explanation of the conjugate gradient method. Seemingly, the algorithm as stated requires storage of all previous searching directions and residue vectors, as well as many matrix–vector multiplications, and thus can be computationally expensive. However, a closer analysis of the algorithm shows that is orthogonal to , i.e. , for i ≠ j. And is -orthogonal to , i.e. , for . This can be regarded that as the algorithm progresses, and span the same Krylov subspace. Where form the orthogonal basis with respect to the standard inner product, and form the orthogonal basis with respect to the inner product induced by . Therefore, can be regarded as the projection of on the Krylov subspace. The algorithm is detailed below for solving Ax = b where is a real, symmetric, positive-definite matrix. The input vector can be an approximate initial solution or 0. It is a different formulation of the exact procedure described above. This is the most commonly used algorithm. The same formula for is also used in the Fletcher–Reeves nonlinear conjugate gradient method. Restarts We note that is computed by the gradient descent method applied to . Setting would similarly make computed by the gradient descent method from , i.e., can be used as a simple implementation of a restart of the conjugate gradient iterations. Restarts could slow down convergence, but may improve stability if the conjugate gradient method misbehaves, e.g., due to round-off error. Explicit residual calculation The formulas and , which both hold in exact arithmetic, make the formulas and mathematically equivalent. The former is used in the algorithm to avoid an extra multiplication by since the vector is already computed to evaluate . The latter may be more accurate, substituting the explicit calculation for the implicit one by the recursion subject to round-off error accumulation, and is thus recommended for an occasional evaluation. A norm of the residual is typically used for stopping criteria. The norm of the explicit residual provides a guaranteed level of accuracy both in exact arithmetic and in the presence of the rounding errors, where convergence naturally stagnates. In contrast, the implicit residual is known to keep getting smaller in amplitude well below the level of rounding errors and thus cannot be used to determine the stagnation of convergence. Computation of alpha and beta In the algorithm, is chosen such that is orthogonal to . The denominator is simplified from since .