analysis via linear models analysis of data from linear regression model is greatly simplified through the linear model representation ( in this section we present the main ideas for parameter estimation and model selection for general linear model of the form xb ( where is an nx matrixb [ ] vector of parametersand [ en ]an -dimensional vector of independent error termswith ei and var ei note that the model matrix is assumed to be fixedand and are random specific outcome of is denoted by (in accordance with the notation in section note that the multiple linear regression model in ( was defined using different parameterizationin particularthere we used [ bd ]sowhen applying the results in the present section to such modelsbe aware that alsoin this section feature vector includes the constant so that [ xn parameter estimation the linear model xb contains two unknown parametersb and which have to be estimated from the training data to estimate bwe can repeat exactly the same reasoning used in our recurring polynomial regression example as follows for linear prediction function (xxbthe (squared-errortraining loss can be written as ` ( ky xbk and the optimal learner gt minimizes this quantityleading to the least-squares estimate bwhich satisfies the normal equations xx xy ( the corresponding training loss can be taken as an estimate of that isc xb ( to justify the latternote that is the second moment of the model errors ei nin ( and could be estimated via the method of moments (see section using the sample average - kek / ky xbk /nif were known by replacing with its estimatorwe arrive at ( note that no distributional properties of the {ei were used other than ei and var ei the vector : xb is called the vector of residuals and approximates the (unknownvector of model errors the quantity kek ni= is called the residual sum of squares (rssdividing the rss by gives an unbiased estimate of which we call the estimated residual squared error (rse)see exercise residuals residual sum of squares residual squared error