nsupervised earning when there is no distinction between response and explanatory variablesunsupervised methods are required to learn the structure of the data in this we look at various unsupervised learning techniquessuch as density estimationclusteringand principal component analysis important tools in unsupervised learning include the cross-entropy training lossmixture modelsthe expectation-maximization algorithmand the singular value decomposition introduction in contrast to supervised learningwhere an "output(responsevariable is explained by an "input(explanatoryvector xin unsupervised learning there is no response variable and the overall goal is to extract useful information and patterns from the datae in the form { xn or as matrix [ xn in essencethe objective of unsupervised learning is to learn about the underlying probability distribution of the data we start in section by setting up framework for unsupervised learning that is similar to the framework used for supervised learning in section that iswe formulate unsupervised learning in terms of risk and loss minimizationbut now involving the crossentropy riskrather than the squared-error risk in natural way this leads to fundamental learning concepts such as likelihoodfisher informationand the akaike information criterion section introduces the expectation-maximization (emalgorithm as useful method for maximizing likelihood functions when their solution cannot be found easily in closed form if the data forms an iid sample from some unknown distributionthe "empirical distributionof the data provides valuable information about the unknown distribution in section we formalize the concept of the empirical distribution ( generalization of the empirical cdfand explain how we can produce an estimate of the underlying probability density function of the data using kernel density estimators most unsupervised learning techniques focus on identifying certain traits of the underlying distributionsuch as its local maximizers related idea is to partition the data into clusters of points that are in some sense "similarto each other in section we formulate the clustering problem in terms of mixture model in particularthe data are assumed