Question

What are some of the similarities between Expectation Maximization and K-mean aside form both being clustering methods.

Answer 1

K means clustering:

K-means clustering is probably one of the first unsupervised learning algorithms that most people encounter when they begin a machine learning course. It is easy to use and intuitive. However, if we indulge in the probabilistic theory behind K-means, it becomes apparent that the algorithm makes very general assumptions regarding the distribution of the data. The K-means algorithm attempts to detect clusters within the dataset under the optimization criteria that the sum of the inter-cluster variances is minimized. TheThe k-means algorithm for partitioning, where each

cluster’s center is represented by the mean value of the objects in the cluster.

Expectation maximization estimation:

Expectation Maximization (EM) is another popular, though a bit more complicated, clustering algorithm that relies on maximizing the likelihood to find the statistical parameters of the underlying sub-populations in the dataset. I will not get into the probabilistic theory behind EM. If you are interested you can read more here. But to briefly summarize, the EM algorithm alternates between two steps (E-step and M-step). In the E-step the algorithm tries to find a lower bound function on the original likelihood using the current estimate of the statistical parameters. In the M-step the algorithm finds new estimates of those statistical parameters by maximizing the lower bound function (i.e. determine the MLE of the statistical parameters).