gausthetakde py import matplotlib pyplot as plt import numpy as np from kde import sig sig sig ** np sqrt ( np pi)/sig constants phi lambda , np exp (-( - ** /( sig )unscaled kernel lambda xnp exp(- )*( > true pdf ** sample size -np log(np random uniform (size= ))generate data via it method xx np arange - , , dtype " ")plot range phis np zeros (len(xx)for in range ( , )phis phis phi(xx , [ ]phis *phis/ plt plot(xx ,phis ,' ')plot gaussian kde [bandwidth ,density ,xmesh ,cdfkde( , ** , max( )idx xmesh < plt plotxmesh [idx]density [idx ])plot theta kde plt plot(xx , (xx))plot true pdf clustering via mixture models clustering is concerned with the grouping of unlabeled feature vectors into clusterssuch that samples within cluster are more similar to each other than samples belonging to different clusters usuallyit is assumed that the number of clusters is known in advancebut otherwise no prior information is given about the data applications of clustering can be found in the areas of communicationdata compression and storagedatabase searchingpattern matchingand object recognition common approach to clustering analysis is to assume that the data comes from mixture of (usually gaussiandistributionsand thus the objective is to estimate the parameters of the mixture model by maximizing the likelihood function for the data direct optimization of the likelihood function in this case is not simple taskdue to necessary constraints on the parameters (more about this laterand the complicated nature of the likelihood functionwhich in general has great number of local maxima and saddle-points popular method to estimate the parameters of the mixture model is the em algorithmwhich was discussed in more general setting in section in this section we explain the basics of mixture modeling and explain the workings of the em method in this context in additionwe show how direct optimization methods can be used to maximize the likelihood mixture models let :{ xn be iid random vectors taking values in some set rd each xi being distributed according to the mixture density ( thw ph (xwk phk ( ) ( mixture density