def lgmres(A, b, x0=None, tol=<NUM_LIT>, maxiter=<NUM_LIT:1000>, M=None, callback=None,<EOL>inner_m=<NUM_LIT:30>, outer_k=<NUM_LIT:3>, outer_v=None, store_outer_Av=True):
from scipy.linalg.basic import lstsq<EOL>A,M,x,b,postprocess = make_system(A,M,x0,b)<EOL>if not np.isfinite(b).all():<EOL><INDENT>raise ValueError("<STR_LIT>")<EOL><DEDENT>matvec = A.matvec<EOL>psolve = M.matvec<EOL>if outer_v is None:<EOL><INDENT>outer_v = []<EOL><DEDENT>axpy, dot, scal = None, None, None<EOL>b_norm = norm2(b)<EOL>if b_norm == <NUM_LIT:0>:<EOL><INDENT>b_norm = <NUM_LIT:1><EOL><DEDENT>for k_outer in xrange(maxiter):<EOL><INDENT>r_outer = matvec(x) - b<EOL>if callback is not None:<EOL><INDENT>callback(x)<EOL><DEDENT>if axpy is None:<EOL><INDENT>if np.iscomplexobj(r_outer) and not np.iscomplexobj(x):<EOL><INDENT>x = x.astype(r_outer.dtype)<EOL><DEDENT>axpy, dot, scal = get_blas_funcs(['<STR_LIT>', '<STR_LIT>', '<STR_LIT>'],<EOL>(x, r_outer))<EOL><DEDENT>r_norm = norm2(r_outer)<EOL>if r_norm < tol * b_norm or r_norm < tol:<EOL><INDENT>break<EOL><DEDENT>vs0 = -psolve(r_outer)<EOL>inner_res_0 = norm2(vs0)<EOL>if inner_res_0 == <NUM_LIT:0>:<EOL><INDENT>rnorm = norm2(r_outer)<EOL>raise RuntimeError("<STR_LIT>"<EOL>"<STR_LIT>" % rnorm)<EOL><DEDENT>vs0 = scal(<NUM_LIT:1.0>/inner_res_0, vs0)<EOL>hs = []<EOL>vs = [vs0]<EOL>ws = []<EOL>y = None<EOL>for j in xrange(<NUM_LIT:1>, <NUM_LIT:1> + inner_m + len(outer_v)):<EOL><INDENT>v_new = None<EOL>if j < len(outer_v) + <NUM_LIT:1>:<EOL><INDENT>z, v_new = outer_v[j-<NUM_LIT:1>]<EOL><DEDENT>elif j == len(outer_v) + <NUM_LIT:1>:<EOL><INDENT>z = vs0<EOL><DEDENT>else:<EOL><INDENT>z = vs[-<NUM_LIT:1>]<EOL><DEDENT>if v_new is None:<EOL><INDENT>v_new = psolve(matvec(z))<EOL><DEDENT>else:<EOL><INDENT>v_new = v_new.copy()<EOL><DEDENT>hcur = []<EOL>for v in vs:<EOL><INDENT>alpha = dot(v, v_new)<EOL>hcur.append(alpha)<EOL>v_new = axpy(v, v_new, v.shape[<NUM_LIT:0>], -alpha)  <EOL><DEDENT>hcur.append(norm2(v_new))<EOL>if hcur[-<NUM_LIT:1>] == <NUM_LIT:0>:<EOL><INDENT>bailout = True<EOL><DEDENT>else:<EOL><INDENT>bailout = False<EOL>v_new = scal(<NUM_LIT:1.0>/hcur[-<NUM_LIT:1>], v_new)<EOL><DEDENT>vs.append(v_new)<EOL>hs.append(hcur)<EOL>ws.append(z)<EOL>if not bailout and j % <NUM_LIT:5> != <NUM_LIT:1> and j < inner_m + len(outer_v) - <NUM_LIT:1>:<EOL><INDENT>continue<EOL><DEDENT>hess = np.zeros((j+<NUM_LIT:1>, j), x.dtype)<EOL>e1 = np.zeros((j+<NUM_LIT:1>,), x.dtype)<EOL>e1[<NUM_LIT:0>] = inner_res_0<EOL>for q in xrange(j):<EOL><INDENT>hess[:(q+<NUM_LIT:2>),q] = hs[q]<EOL><DEDENT>y, resids, rank, s = lstsq(hess, e1)<EOL>inner_res = norm2(np.dot(hess, y) - e1)<EOL>if inner_res < tol * inner_res_0:<EOL><INDENT>break<EOL><DEDENT><DEDENT>dx = ws[<NUM_LIT:0>]*y[<NUM_LIT:0>]<EOL>for w, yc in zip(ws[<NUM_LIT:1>:], y[<NUM_LIT:1>:]):<EOL><INDENT>dx = axpy(w, dx, dx.shape[<NUM_LIT:0>], yc)  <EOL><DEDENT>nx = norm2(dx)<EOL>if store_outer_Av:<EOL><INDENT>q = np.dot(hess, y)<EOL>ax = vs[<NUM_LIT:0>]*q[<NUM_LIT:0>]<EOL>for v, qc in zip(vs[<NUM_LIT:1>:], q[<NUM_LIT:1>:]):<EOL><INDENT>ax = axpy(v, ax, ax.shape[<NUM_LIT:0>], qc)<EOL><DEDENT>outer_v.append((dx/nx, ax/nx))<EOL><DEDENT>else:<EOL><INDENT>outer_v.append((dx/nx, None))<EOL><DEDENT>while len(outer_v) > outer_k:<EOL><INDENT>del outer_v[<NUM_LIT:0>]<EOL><DEDENT>x += dx<EOL><DEDENT>else:<EOL><INDENT>return postprocess(x), maxiter<EOL><DEDENT>return postprocess(x), <NUM_LIT:0><EOL>
Solve a matrix equation using the LGMRES algorithm.

The LGMRES algorithm [BJM]_ [BPh]_ is designed to avoid some problems
in the convergence in restarted GMRES, and often converges in fewer
iterations.

Parameters
----------
A : {sparse matrix, dense matrix, LinearOperator}
    The real or complex N-by-N matrix of the linear system.
b : {array, matrix}
    Right hand side of the linear system. Has shape (N,) or (N,1).
x0  : {array, matrix}
    Starting guess for the solution.
tol : float
    Tolerance to achieve. The algorithm terminates when either the relative
    or the absolute residual is below `tol`.
maxiter : int
    Maximum number of iterations.  Iteration will stop after maxiter
    steps even if the specified tolerance has not been achieved.
M : {sparse matrix, dense matrix, LinearOperator}
    Preconditioner for A.  The preconditioner should approximate the
    inverse of A.  Effective preconditioning dramatically improves the
    rate of convergence, which implies that fewer iterations are needed
    to reach a given error tolerance.
callback : function
    User-supplied function to call after each iteration.  It is called
    as callback(xk), where xk is the current solution vector.
inner_m : int, optional
    Number of inner GMRES iterations per each outer iteration.
outer_k : int, optional
    Number of vectors to carry between inner GMRES iterations.
    According to [BJM]_, good values are in the range of 1...3.
    However, note that if you want to use the additional vectors to
    accelerate solving multiple similar problems, larger values may
    be beneficial.
outer_v : list of tuples, optional
    List containing tuples ``(v, Av)`` of vectors and corresponding
    matrix-vector products, used to augment the Krylov subspace, and
    carried between inner GMRES iterations. The element ``Av`` can
    be `None` if the matrix-vector product should be re-evaluated.
    This parameter is modified in-place by `lgmres`, and can be used
    to pass "guess" vectors in and out of the algorithm when solving
    similar problems.
store_outer_Av : bool, optional
    Whether LGMRES should store also A*v in addition to vectors `v`
    in the `outer_v` list. Default is True.

Returns
-------
x : array or matrix
    The converged solution.
info : int
    Provides convergence information:

        - 0  : successful exit
        - >0 : convergence to tolerance not achieved, number of iterations
        - <0 : illegal input or breakdown

Notes
-----
The LGMRES algorithm [BJM]_ [BPh]_ is designed to avoid the
slowing of convergence in restarted GMRES, due to alternating
residual vectors. Typically, it often outperforms GMRES(m) of
comparable memory requirements by some measure, or at least is not
much worse.

Another advantage in this algorithm is that you can supply it with
'guess' vectors in the `outer_v` argument that augment the Krylov
subspace. If the solution lies close to the span of these vectors,
the algorithm converges faster. This can be useful if several very
similar matrices need to be inverted one after another, such as in
Newton-Krylov iteration where the Jacobian matrix often changes
little in the nonlinear steps.

References
----------
.. [BJM] A.H. Baker and E.R. Jessup and T. Manteuffel,
         SIAM J. Matrix Anal. Appl. 26, 962 (2005).
.. [BPh] A.H. Baker, PhD thesis, University of Colorado (2003).
         http://amath.colorado.edu/activities/thesis/allisonb/Thesis.ps