regression, the hat matrix H is X(X 'X)−1X ', and all these definitions reduce to the usual degrees of freedom. Notice that the regression (not residual) degrees of freedom in linear models are "the sum of the sensitivities of the fitted values with respect to the observed response values", i.e. the sum of leverage scores. One way to help to conceptualize this is to consider a simple smoothing matrix like a Gaussian blur, used to mitigate data noise. In contrast to a simple linear or polynomial fit, computing the effective degrees of freedom of the smoothing function is not straight-forward. In these cases, it is important to estimate the Degrees of Freedom permitted by the matrix so that the residual degrees of freedom can then be used to estimate statistical tests such as . Residual effective degrees of freedom There are corresponding definitions of residual effective degrees-of-freedom (redf), with H replaced by I − H. For example, if the goal is to estimate error variance, the redf would be defined as tr((I − H)'(I − H)), and the unbiased estimate is (with ), or:Trevor Hastie, Robert Tibshirani (1990), Generalized additive models, CRC Press, (p. 54) and (eq.(B.1), p. 305))David Ruppert, M. P. Wand, R. J. Carroll (2003), Semiparametric Regression, Cambridge University Press (eq.(3.28), p. 82) The last approximation above reduces the computational cost from O(n2) to only O(n). In general the numerator would be the objective function being minimized; e.g., if the hat matrix includes an observation covariance matrix, Σ, then becomes . General Note that unlike in the original case, non-integer degrees of freedom are allowed, though the value must usually still be constrained between 0 and n. Consider, as an example, the k-nearest neighbour smoother, which is the average of the k nearest measured values to the given point. Then, at each of the n measured points, the weight of the original value on the linear combination that makes up the predicted value is just 1/k. Thus, the trace of the hat matrix is n/k. Thus the smooth costs n/k effective degrees of freedom. As another example, consider the existence of nearly duplicated observations. Naive application of classical formula, n − p, would lead to over-estimation of the residuals degree of freedom, as if each observation were independent. More realistically, though, the hat matrix would involve an observation covariance matrix Σ indicating the non-zero correlation among observations. The