the positive definiteness. In order to maintain the symmetry and positive definiteness of , the update form can be chosen as . Imposing the secant condition, . Choosing and , we can obtain: Finally, we substitute and into and get the update equation of : Algorithm From an initial guess and an approximate Hessian matrix the following steps are repeated as converges to the solution: Obtain a direction by solving . Perform a one-dimensional optimization (line search) to find an acceptable stepsize in the direction found in the first step. If an exact line search is performed, then . In practice, an inexact line search usually suffices, with an acceptable satisfying Wolfe conditions. Set and update . . . denotes the objective function to be minimized. Convergence can be checked by observing the norm of the gradient, . If is initialized with , the first step will be equivalent to a gradient descent, but further steps are more and more refined by , the approximation to the Hessian. The first step of the algorithm is carried out using the inverse of the matrix , which can be obtained efficiently by applying the Sherman–Morrison formula to the step 5 of the algorithm, giving This can be computed efficiently without temporary matrices, recognizing that is symmetric, and that and are scalars, using an expansion such as Therefore, in order to avoid any matrix inversion, the inverse of the Hessian can be approximated instead of the Hessian itself: From an initial guess and an approximate inverted Hessian matrix the following steps are repeated as converges to the solution: Obtain a direction by solving . Perform a one-dimensional optimization (line search) to find an acceptable stepsize in the direction found in the first step. If an exact line search is performed, then . In practice, an inexact line search usually suffices, with an acceptable satisfying Wolfe conditions. Set and update . . . In statistical estimation problems (such as maximum likelihood or Bayesian inference), credible intervals or confidence intervals for the solution can be estimated from the inverse of the final Hessian matrix . However, these quantities are technically defined by the true Hessian matrix, and the BFGS approximation may not converge to the true Hessian matrix. Notable implementations Notable open source implementations are: ALGLIB implements BFGS and its limited-memory version in C++ and C# GNU Octave uses a form of BFGS in its