and where and are vectors with -th component and respectively. The above expression obtained for comes under the Gauss–Newton method. The Jacobian matrix as defined above is not (in general) a square matrix, but a rectangular matrix of size , where is the number of parameters (size of the vector ). The matrix multiplication yields the required square matrix and the matrix-vector product on the right hand side yields a vector of size . The result is a set of linear equations, which can be solved for . Levenberg's contribution is to replace this equation by a "damped version": where is the identity matrix, giving as the increment to the estimated parameter vector . The (non-negative) damping factor is adjusted at each iteration. If reduction of is rapid, a smaller value can be used, bringing the algorithm closer to the Gauss–Newton algorithm, whereas if an iteration gives insufficient reduction in the residual, can be increased, giving a step closer to the gradient-descent direction. Note that the gradient of with respect to equals . Therefore, for large values of , the step will be taken approximately in the direction opposite to the gradient. If either the length of the calculated step or the reduction of sum of squares from the latest parameter vector fall below predefined limits, iteration stops, and the last parameter vector is considered to be the solution. When the damping factor is large relative to , inverting is not necessary, as the update is well-approximated by the small gradient step . To make the solution scale invariant Marquardt's algorithm solved a modified problem with each component of the gradient scaled according to the curvature. This provides larger movement along the directions where the gradient is smaller, which avoids slow convergence in the direction of small gradient. Fletcher in his 1971 paper A modified Marquardt subroutine for non-linear least squares simplified the form, replacing the identity matrix with the diagonal matrix consisting of the diagonal elements of : A similar damping factor appears in Tikhonov regularization, which is used to solve linear ill-posed problems, as well as in ridge regression, an estimation technique in statistics. Choice of damping parameter Various more or less heuristic arguments have been put forward for the best choice for the damping parameter . Theoretical arguments exist showing why some of these choices guarantee local convergence of the algorithm; however, these choices can make the