a statistical approach, and by Manus Foster, who interpreted this method as a Wiener–Kolmogorov (Kriging) filter. Following Hoerl, it is known in the statistical literature as ridge regression, named after the shape along the diagonal of the identity matrix. Tikhonov regularization Suppose that for a known matrix and vector , we wish to find a vector such that The standard approach is ordinary least squares linear regression. However, if no satisfies the equation or more than one does—that is, the solution is not unique—the problem is said to be ill posed. In such cases, ordinary least squares estimation leads to an overdetermined, or more often an underdetermined system of equations. Most real-world phenomena have the effect of low-pass filters in the forward direction where maps to . Therefore, in solving the inverse-problem, the inverse mapping operates as a high-pass filter that has the undesirable tendency of amplifying noise (eigenvalues / singular values are largest in the reverse mapping where they were smallest in the forward mapping). In addition, ordinary least squares implicitly nullifies every element of the reconstructed version of that is in the null-space of , rather than allowing for a model to be used as a prior for . Ordinary least squares seeks to minimize the sum of squared residuals, which can be compactly written as where is the Euclidean norm. In order to give preference to a particular solution with desirable properties, a regularization term can be included in this minimization: for some suitably chosen Tikhonov matrix . In many cases, this matrix is chosen as a scalar multiple of the identity matrix (), giving preference to solutions with smaller norms; this is known as regularization. In other cases, high-pass operators (e.g., a difference operator or a weighted Fourier operator) may be used to enforce smoothness if the underlying vector is believed to be mostly continuous. This regularization improves the conditioning of the problem, thus enabling a direct numerical solution. An explicit solution, denoted by , is given by The effect of regularization may be varied by the scale of matrix . For this reduces to the unregularized least-squares solution, provided that (ATA)−1 exists. regularization is used in many contexts aside from linear regression, such as classification with logistic regression or support vector machines, and matrix factorization. Generalized Tikhonov regularization For general multivariate normal distributions for and the data error, one can apply a transformation of the