In statistics, generalized least squares (GLS) is a technique for estimating the unknown parameters in a linear regression model when there is a certain degree of correlation between the residuals in a regression model. In these cases, ordinary least squares and weighted least squares can be statistically inefficient, or even give misleading inferences. GLS was first described by Alexander Aitken in 1936. Method outline In standard linear regression models we observe data on n statistical units. The response values are placed in a vector , and the predictor values are placed in the design matrix , where is a vector of the k predictor variables (including a constant) for the ith unit. The model forces the conditional mean of given to be a linear function of , and assumes the conditional variance of the error term given is a known nonsingular covariance matrix . This is usually written as Here is a vector of unknown constants (known as “regression coefficients”) that must be estimated from the data. Suppose is a candidate estimate for . Then the residual vector for will be . The generalized least squares method estimates by minimizing the squared Mahalanobis length of this residual vector: where the last two terms evaluate to scalars, resulting in This objective is a quadratic form in . Taking the gradient of this quadratic form with respect to and equating it to zero (when ) gives Therefore, the minimum of the objective function can be computed yielding the explicit formula: The quantity is known as the precision matrix (or dispersion matrix), a generalization of the diagonal weight matrix. Properties The GLS estimator is unbiased, consistent, efficient, and asymptotically normal with and . GLS is equivalent to applying ordinary least squares to a linearly transformed version of the data. To see this, factor , for instance using the Cholesky decomposition. Then if we pre-multiply both sides of the equation by , we get an equivalent linear model where , , and . In this model , where is the identity matrix. Thus we can efficiently estimate by applying Ordinary least squares (OLS) to the transformed data, which requires minimizing This has the effect of standardizing the scale of the errors and “de-correlating” them. Since OLS is applied to data with homoscedastic errors, the Gauss–Markov theorem applies, and therefore the GLS estimate is the best linear unbiased estimator for β. Weighted least squares A