the Gauss–Newton algorithm can approach quadratic. The algorithm may converge slowly or not at all if the initial guess is far from the minimum or the matrix is ill-conditioned. For example, consider the problem with equations and variable, given by The optimum is at . (Actually the optimum is at for , because , but .) If , then the problem is in fact linear and the method finds the optimum in one iteration. If |λ| < 1, then the method converges linearly and the error decreases asymptotically with a factor |λ| at every iteration. However, if |λ| > 1, then the method does not even converge locally. Solving overdetermined systems of equations The Gauss-Newton iteration is an effective method for solving overdetermined systems of equations in the form of with and where is the Moore-Penrose inverse (also known as pseudoinverse) of the Jacobian matrix of . It can be considered an extension of Newton's method and enjoys the same local quadratic convergence toward isolated regular solutions. If the solution doesn't exist but the initial iterate is near a point at which the sum of squares reaches a small local minimum, the Gauss-Newton iteration linearly converges to . The point is often called a least squares solution of the overdetermined system. Derivation from Newton's method In what follows, the Gauss–Newton algorithm will be derived from Newton's method for function optimization via an approximation. As a consequence, the rate of convergence of the Gauss–Newton algorithm can be quadratic under certain regularity conditions. In general (under weaker conditions), the convergence rate is linear. The recurrence relation for Newton's method for minimizing a function S of parameters is where g denotes the gradient vector of S, and H denotes the Hessian matrix of S. Since , the gradient is given by Elements of the Hessian are calculated by differentiating the gradient elements, , with respect to : The Gauss–Newton method is obtained by ignoring the second-order derivative terms (the second term in this expression). That is, the Hessian is approximated by where are entries of the Jacobian Jr. Note that when the exact hessian is evaluated near an exact fit we have near-zero , so the second term becomes near-zero as well, which justifies the approximation. The gradient and the approximate Hessian can be written in matrix notation as These expressions are substituted into the recurrence relation above to obtain the operational equations