figure illustrates the modeling tradeoffs the keywords within the triangle represent various modeling paradigms few keywords have been highlightedsymbolizing their importance in modeling the specific meaning of the keywords does not concern us herebut the point is there are many models to choose fromdepending on what assumptions are made about the data figure illustration of the modeling dilemma complex models are more generally applicablebut may be difficult to analyze simple models may be highly tractablebut may not describe the data accurately the triangular shape signifies that there are great many specific models but not so many generic ones on the one handmodels that make few assumptions are more widely applicablebut at the same time may not be very mathematically tractable or provide insight into the nature of the data on the other handvery specific models may be easy to handle and interpretbut may not match the data very well this tradeoff between the tractability and applicability of the model is very similar to the approximation-estimation tradeoff described in section in the typical unsupervised setting we have training set { xn that is viewed as the outcome of iid random variables xn from some unknown pdf the objective is then to learn or estimate from the finite training data to put the learning in similar framework as for supervised learning discussed in the preceding sections we begin by specifying class of probability density functions :{ (th)th th}where th is parameter in some subset th of we now seek the best in to minimize some risk note that may not necessarily contain the true even for very large we stress that our notation (xhas different meaning in the supervised and unsupervised case in the supervised caseg is interpreted as prediction function for response yin the unsupervised settingg is an approximation of density for each we measure the discrepancy between the true model (xand the hypothesized model ( thusing the loss function lossf ( ) ( th)ln (xln (xln ( thg( th