in terms of the notation given in the summary table for supervised learningwe thus have the (observedtraining data is {xy the function class is the class of linear functions of xthat is { (bx xbb the (squared-errortraining loss is ` ( ( )ky xbk / bwhere argminbr ky xbk the learner gt is given by gt (xx> the minimal training loss is ` (gt ky xb bk / model selection and prediction even if we restrict the learner to be linear functionthere is still the issue of which explanatory variables (featuresto include while including too few features may result in large approximation error (underfitting)including too many may result in large statistical error (overfittingas discussed in section we need to select the features which provide the best tradeoff between the approximation and statistical errorsso that the (expectedgeneralization risk of the learner is minimized depending on how the (expectedgeneralization risk is estimatedthere are number of strategies for feature selection use test data ( that are obtained independently from the training data tto estimate the generalization risk ky gt ( ) via the test loss ( then choose the collection of features that minimizes the test loss when there is an abundance of datapart of the data can be reserved as test datawhile the remaining data is used as training data when there is limited amount of datawe can use cross-validation to estimate the expected generalization risk ky gt ( ) (where is random training set)as explained in section this is then minimized over the set of possible choices for the explanatory variables when one has to choose between many potential explanatory variablestechniques such as regularized least-squares and lasso regression become important such methods offer another approach to model selectionvia the regularization (or homotopypaths this will be the topic of section in the next rather than using computer-intensive techniquessuch as the ones aboveone can use theoretical estimates of the expected generalization risksuch as the in-sample riskaicand bicas in section and minimize this to determine good set of explanatory variables all of the above approaches do not assume any distributional properties of the error terms {ei in the linear modelother than that they are independent with expectation and variance ifhoweverthey are assumed to have normal (gaussiandistribution(that is{ei ~iid ( ))then the inclusion and exclusion of variables can