bayesian learning in bayesian unsupervised learningwe seek to approximate the unknown joint density ( xn of the training data tn { xn via joint pdf of the form (xi thw(thdth( = where (thbelongs to family of parametric densities :{ (th)th th(viewed as family of pdfs conditional on parameter th in some set th and (this pdf that belongs to (possibly differentfamily of densities note how the joint pdf ( satisfies the permutational invariance ( and can thus be useful as model for training data which is part of an exchangeable sequence of random variables following standard practice in bayesian contextinstead of writing fx (xand fx ( yfor the pdf of and the conditional pdf of given yone simply writes (xand ( yif is different random variableits pdf (at yis thus denoted by (ythuswe will use the same symbol for different (conditionalapproximating probability densities and for the different (conditionaltrue and unknown probability densities using bayesian notationwe can writerg( thni= (xi thand thus the approximating joint pdf ( can then be written as ( thw(thdth and the true unknown joint pdf as (tf ( xn once and are specifiedselecting an approximating function (xof the form (xg( thw(thdth is equivalent to selecting suitable from similar to ( )we can use the kullbackleibler risk to measure the discrepancy between the proposed approximation ( and the true ( ) (tf ( (tln dt ( `(ge ln ( thw(thdth ( thw(thdth the main difference with ( is that since the training data is not necessarily iid (it may be exchangeablefor example)the expectation must be with respect to the joint density of not with respect to the marginal ( (as in the iid caseminimizing the training loss is equivalent to maximizing the likelihood of the training data tthat issolving the optimization problem max ( thw(thdthww where the maximization is over an appropriate class of density functions that is believed to result in the smallest kl risk