example (polynomial regression (cont )we continue example where the components of the response vector [ yn ]are independent and normally distributed with variance ` (the irreducible errorand expectations ex yi (xi > bi using the formula ( for the least-squares estimator bthe expected optimism ( is   covx > byi tr covx xb by tr covx xxyy = tr (xxcovx (yy) `tr (xx ` in the last equation we used the cyclic property of the trace (theorem )tr(xxtr(xxtr( )assuming that rank(xp thereforean estimate for the in-sample risk ( isb `in (gt ` (gt ` / ( where we have assumed that the irreducible risk `is known figure shows that this estimate is very close to the test loss from figure henceinstead of computing the test loss to assess the best model complexity pwe could simply have minimized the training loss plus the correction term ` / in practice`also has to be estimated somehow figure in-sample risk estimate `in (gt as function of the number of parameters of the model the test loss is superimposed as blue dashed curve cross-validation in generalfor complex function classes git is very difficult to derive simple formulas of the approximation and statistical errorslet alone for the generalization risk or expected generalization risk as we sawwhen there is an abundance of datathe easiest way to assess the generalization risk for given training set is to obtain test set and evaluate the test loss ( when sufficiently large test set is not available but computational resources are cheapone can instead gain direct knowledge of the expected generalization risk via computationally intensive method called cross-validation cross-validation