so it suffices to specify the discrete pdfp(ti sayof each zi given the observed point xi the latter can be found from bayesformula( - (tphk (xi ( - ( - ) (kwk ( nextin view of ( )the function ( (thcan be written as (tq (the (te (tln wzi ln phzi (xi uzi szi ln wzi ln phzi (xi uzi szi = = where the {zi are independent and zi is distributed according to (ti in ( this com(tpletes the -step in the -step we maximize with respect to the parameter ththat iswith respect to the {wk }{uk }and {sk in particularwe maximize = = (ti (kln wk ln phk (xi uk sk pk (tunder the condition wk using lagrange multipliers and the fact that = pi ( gives the solution for the {wk } (tp ( ) ( wk = the solutions for uk and sk now follow from maximizing ni= (ti (kln phk (xi uk sk )leading to pn (ti= (kxi ( uk pn (tp (ki= and pn (ti= pi ( (xi uk )(xi uk ( sk pn (tp (ki= which are very similar to the well-known formulas for the mles of the parameters of gaussian distribution after assigning the solution parameters to th(tand increasing the iteration counter by the steps ( )( )( )and ( are repeated until convergence is reached convergence of the em algorithm is very sensitive to the choice of initial parameters it is therefore recommended to try various different starting conditions for further discussion of the theoretical and practical aspects of the em algorithm we refer to [ example (clustering via emwe return to the data in example depicted in figure and adopt the model that the data is coming from mixture of three bivariate gaussian distributions the python code below implements the em procedure described in algorithm the initial mean vectors {uk of the bivariate gaussian distributions are chosen (from visual inspectionto lie roughly in the middle of each clusterin this case [- - ][- ]and [ - ]the corresponding covariance matrices are initially chosen as identity matriceswhich is appropriate given the observed spread of the data in figure finallythe initial weights are / / / for simplicitythe algorithm stops after iterationswhich in this case is more than enough to guarantee convergence the code and data are available from the book' website in the github folder