th exp(- th) th in other wordswe look for the best gg within the family of exponential distributions with unknown parameter th the likelihood of the data is = (xi thn th exp(-thxi exp(-th xn ln thi= and the score is ( th- th- thusmaximizing the likelihood with respect to th is the same as maximizing -th xn ln th or solving ni= (xi th)/ xn th- in other wordsthe solution to ( is the maximum likelihood estimate thn /xn in supervised settingwhere the data is represented by vector of explanatory variables and response ythe general model is that (xyis an outcome of (xyf for some unknown and for training sequence ( )(xn yn the default model assumption is that ( )(xn yn ~iid as explained in section the analysis primarily involves the conditional pdf ( xand in particular (when using the squarederror lossthe conditional expectation (xe[ xthe resulting representation ( allows us to then write the response at as function of the feature plus an error termy (xe(xthis leads to the simplest and most important model for supervised learningwhere we choose linear class of prediction or guess functions and assume that it is rich enough to contain the true gif we further assume thatconditional on xthe error term does not depend on xthat ise and var then we obtain the following model definition linear model in linear model the response depends on -dimensional explanatory variable [ ]via the linear relationship xb elinear model ( where and var note that ( is model for single pair (xythe model for the training set {(xi yi )is simply that each yi satisfies ( (with xi and that the {yi are independent gathering all responses in the vector [ yn ]we can write xb ( where [ en ]is vector of iid copies of and is the so-called model matrixwith rows > > linear models are fundamental building blocks of statistical learning algorithms for this reasona large part of is devoted to linear regression models example (polynomial regression (cont )for our running example we see that the data is described by linear model of the form ( )with model matrix given in ( model matrix