helpful in statistics, for example, in establishing the degree of bias in using a sample variance as an estimate of a population variance). Relationships between idempotent matrices In regression analysis, the matrix is known to produce the residuals from the regression of the vector of dependent variables on the matrix of covariates . (See the section on Applications.) Now, let be a matrix formed from a subset of the columns of , and let . It is easy to show that both and are idempotent, but a somewhat surprising fact is that . This is because , or in other words, the residuals from the regression of the columns of on are 0 since can be perfectly interpolated as it is a subset of (by direct substitution it is also straightforward to show that ). This leads to two other important results: one is that is symmetric and idempotent, and the other is that , i.e., is orthogonal to . These results play a key role, for example, in the derivation of the F test. Any similar matrices of an idempotent matrix are also idempotent. Idempotency is conserved under a change of basis. This can be shown through multiplication of the transformed matrix with being idempotent: . Applications Idempotent matrices arise frequently in regression analysis and econometrics. For example, in ordinary least squares, the regression problem is to choose a vector of coefficient estimates so as to minimize the sum of squared residuals (mispredictions) ei: in matrix form, Minimize where is a vector of dependent variable observations, and is a matrix each of whose columns is a column of observations on one of the independent variables. The resulting estimator is where superscript T indicates a transpose, and the vector of residuals is Here both and (the latter being known as the hat matrix) are idempotent and symmetric matrices, a fact which allows simplification when the sum of squared residuals is computed: The idempotency of plays a role in other calculations as well, such as in determining the variance of the estimator . An idempotent linear operator is a projection operator on the range space along its null space . is an orthogonal projection operator if and only if it is idempotent and symmetric. See also Idempotence Nilpotent Projection (linear algebra) Hat matrix References Algebra Regression analysis Matrices