expectation-maximization (emalgorithm the expectation-maximization algorithm (emis general algorithm for maximization of complicated (log-)likelihood functionsthrough the introduction of auxiliary variables to simplify the notation in this sectionwe use bayesian notation systemwhere the same symbol is used for different (conditionalprobability densities as in the previous sectiongiven independent observations { xn from some unknown pdf the objective is to find the best approximation to in function class { (th)th thby solving the maximum likelihood problemthargmax ( th)thth latent variables complete-data likelihood ( where ( th: ( thg(xn ththe key element of the em algorithm is the augmentation of the data with suitable vector of latent variableszsuch that ( thg(tz thdz the function th (tz this usually referred to as the complete-data likelihood function the choice of the latent variables is guided by the desire to make the maximization of (tz thmuch easier than that of ( thsuppose denotes an arbitrary density of the latent variables thenwe can writez ln ( thp(zln ( thdz (tz th)/ (zdz (zln ( tth)/ (zz (tz thg( tthdz (zln dz (zln (zp(zz (tz thp(zln dz (pg(tth))( (zwhere (pg(tth)is the kullback-leibler divergence from the density to (tthsince it follows that (tz thln ( thp(zln dz = (pthp(zfor all th and any density of the latent variables in other wordsl(pthis lower bound on the log-likelihood that involves the complete-data likelihood the em algorithm then aims to increase this lower bound as much as possible by starting with an initial guess th( and thenfor solving the following two steps (targmax (pth( - ) th(targmaxthth ( (tth