table summary of definitions for unsupervised learning (xt or tn or tn lossf ( ) ( )`(ggg ` ( ` (gggt or gt ggt or gt fixed feature vector random feature vector pdf of evaluated at the point fixed training data {xi nrandom training data {xi napproximation of the pdf loss incurred when approximating (xwith (xrisk for approximation function gthat ise lossf ( ) ( )optimal approximation function in function class gthat isargmingg `(gtraining loss for approximation function (guessgthat isthe sample average estimate of `(gbased on fixed training sample the same as ` ( )but now for random training sample the learnerargmingg ` (gthat isthe optimal approximation function based on fixed training set and function class we suppress the superscript if the function class is implicit the learner for random training set depends on insteadwe seek to minimize the cross-entropy training loss` ( : lossf (xi ) (xi )ln (xi = = cross-entropy training loss ( over the class of functions gwhere { xn is an iid sample from this optimization is doable without knowing and is equivalent to solving the maximization problem max gg ln (xi ( = key step in setting up the learning procedure is to select suitable function class over which to optimize the standard approach is to parameterize with parameter th and let be the class of functions { (th)th thfor some -dimensional parameter set th for the remainder of section we will be using this function classas well as the cross-entropy risk the function th ( this called the likelihood function it gives the likelihood of the observed feature vector under (th)as function of the parameter th the natural logarithm of the likelihood function is called the log-likelihood function and its gradient with respect to th is called the score functiondenoted ( th)that isg( thln ( ths( th:th th ( th( likelihood function score function