weights where ph phk are probability densities (discrete or continuouson xand the positive weights wk sum up to this mixture pdf can be interpreted in the following way let be discrete random variable taking values with probabilities wk and let be random vector whose conditional pdfgiven zis phz by the product rule ( )the joint pdf of and is given by phz, (zxphz (zph ( zwz phz (xand the marginal pdf of is found by summing the joint pdf over the values of zwhich gives ( random vector can thus be simulated in two steps firstdraw according to the probabilities [ zwz then draw according to the pdf phz as only contain the {xi variablesthe {zi are viewed as latent variables we can interpret zi as the hidden label of the cluster to which xi belongs typicallyeach phk in ( is assumed to be known up to some parameter vector ek it is customary in clustering analysis to work with gaussian mixturesthat iseach density phk is gaussian with some unknown expectation vector uk and covariance matrix sk we gather all unknown parametersincluding the weights {wk }into parameter vector th as usualt { xn denotes the outcome of as the components of are iidtheir (jointpdf is given by ( th: = (xi thn = = wk phk (xi uk sk ( following the same reasoning as for ( )we can estimate th from an outcome by maximizing the log-likelihood function (th :ln (xi thln wk phk (xi uk sk ( = = = howeverfinding the maximizer of (th tis not easy in generalsince the function is typically multiextremal example (clustering via mixture modelsthe data depicted in figure consists of data points that were independently generated from three bivariate normal distributionswhose parameters are given in that same figure for each of these three distributionsexactly points were generated ideallywe would like to cluster the data into three clusters that correspond to the three cases to cluster the data into three groupsa possible model for the data is to assume that the points are iid draws from an (unknownmixture of three -dimensional gaussian distributions this is sensible approachalthough in reality the data were not simulated in this way it is instructive to understand the difference between the two models in the mixture modeleach cluster label takes the value { with equal probabilityand hencedrawing the labels independentlythe total number of points in each cluster would other common mixture distributions include student and beta distributions