only the observed part of the complete random data ( )which were generated via the two-step procedure described above that isfor each data point xfirst draw the cluster label { kaccording to probabilities { wk and thengiven zdraw from phz the joint pdf of and is (tz thcomplete-data log-likelihood wzi phzi (xi ) = which is of much simpler form than ( it follows that the complete-data loglikelihood function ln[wzi phzi (xi )( (th tzi= is often easier to maximize than the original log-likelihood ( )for any given (tzbutof course the latent variables are not observed and so (th tzcannot be evaluated in the -step of the em algorithmthe complete-data log-likelihood is replaced with the expectation (th tz)where the subscript in the expectation indicates that is distributed according to the conditional pdf of given tthat iswith pdf (zg( tthg(tz th( note that (zis of the form ( pn (zn so thatgiven tthe components of are independent of each other the em algorithm for mixture models can now be formulated as follows algorithm em algorithm for mixture models inputdata tinitial guess th( outputapproximation of the maximum likelihood estimate while stopping criterion is not met do expectation stepfind ( ( : ( tth( - and ( (th: ( ) (th tz( ( maximization steplet th argmaxth (th - + return th(ta possible termination condition is to stop when (th(ttl(th( - tl(th(tte for some small tolerance as was mentioned in section the sequence of loglikelihood values does not decrease with each iteration under certain continuity conditionsthe sequence {th(tis guaranteed to converge to local maximizer of the loglikelihood convergence to global maximizer (if it existsdepends on the appropriate choice for the starting value typicallythe algorithm is run from different random starting points for the case of gaussian mixtureseach phk ph(uk sk ) is the density of -dimensional gaussian distribution let th( - be the current guess for the optimal parameter vectorconsisting of the weights { ( - }mean vectors { ( - }and covariance ( - (tmatrices {sk we first determine -the pdf of conditional on -for the given guess th( - as mentioned beforethe components of given are independent