nested models nested models let be model matrix of the form [ ]where and are model matrices of dimension and ( )respectively the linear models and are said to be nested within the linear model xb this simply means that certain features in are ignored in each of the first two models note that bb and are parameter vectors of dimension pkand krespectively in what followswe assume that and that all model matrices are full-rank suppose we wish to assess whether to use the full model matrix or the reduced model matrix let be the estimate of under the full model (that isobtained via ( ))and let bb denote the estimate of for the reduced model let ( xb be the projection of ( onto the space span(xspanned by the columns of xand let bb be the projection of onto the space span( spanned by the columns of onlysee figure in order to decide whether the features in are neededwe may compare the estimated error terms of the two modelsas calculated by ( )that isby the residual sum of squares divided by the number of observations if the outcome of this comparison is that there is little difference between the model error for the full and reduced modelthen it is appropriate to adopt the reduced modelas it has fewer parameters than the full modelwhile explaining the data just as well the comparison is thus between the squared norms ky ( and ky ( because of the nested nature of the linear modelsspan( is subspace of span(xandconsequentlythe orthogonal projection of ( onto span( is the same as the orthogonal projection of onto span( )that isy ( by pythagorastheoremwe thus have the decomposition ky ( ( ky ( ky ( this is also illustrated in figure ( ( ( ( ( span(xy ( span( figure the residual sum of squares for the full model corresponds to ky - ( and for the reduced model it is ky - ( by pythagoras' theoremthe difference is ky ( - ( the above decomposition can be generalized to more than two model matrices suppose that the model matrix can be decomposed into submatricesx [ xd ]where the matrix xi has pi columns and rowsi thusthe number of columns as alwayswe assume the columns are linearly independent