weight - - - - - - - mean vector - - - - - covariance matrix - - figure the results of the em clustering algorithm applied to the data depicted in figure as an alternative to the em algorithmone can of course use continuous multiextremal optimization algorithms to directly optimize the log-likelihood function (th tln ( thin ( over the set th of all possible th this is done for example in [ ]demonstrating superior results to em when there are few data points closer investigation of the likelihood function reveals that there is hidden problem with any maximum likelihood approach for clustering if th is chosen as large as possible - any mixture distribution is possible to demonstrate this problemconsider figure depicting the probability density functiong(thof mixture of two gaussian distributionswhere th [wu ]is the vector of parameters for the mixture distribution the log-likelihood function is given by (th = ln (xi th)where are the data (indicated by dots in the figure - - figure mixture of two gaussian distributions it is clear that by fixing the mixing constant at (sayand centering the first cluster at one can obtain an arbitrarily large likelihood value by taking the variance of the first cluster to be arbitrarily small similarlyfor higher dimensional databy choosing "pointor "lineclustersor in general "degenerateclustersone can make the value of the likelihood infinite this is manifestation of the familiar overfitting problem for the