class_docstr: Implements the L-BFGS algorithm. Compatible with multi-batch and full-overlap
  L-BFGS implementations and (stochastic) Powell damping. Partly based on the
  original L-BFGS implementation in PyTorch, Mark Schmidt's minFunc MATLAB code,
  and Michael Overton's weak Wolfe line search MATLAB code.
  
  Implemented by: Hao-Jun Michael Shi and Dheevatsa Mudigere
  Last edited 12/6/18.
  
  Warnings:
    . Does not support per-parameter options and parameter groups.
    . All parameters have to be on a single device.
  
  Inputs:
      lr (float): steplength or learning rate (default: 1)
      history_size (int): update history size (default: 10)
      line_search (str): designates line search to use (default: 'Wolfe')
          Options:
              'None': uses steplength designated in algorithm
              'Armijo': uses Armijo backtracking line search
              'Wolfe': uses Armijo-Wolfe bracketing line search
      dtype: data type (default: torch.float)
      debug (bool): debugging mode
  
  References:
  [1] Berahas, Albert S., Jorge Nocedal, and Martin Takác. "A Multi-Batch L-BFGS
      Method for Machine Learning." Advances in Neural Information Processing
      Systems. 2016.
  [2] Bollapragada, Raghu, et al. "A Progressive Batching L-BFGS Method for Machine
      Learning." International Conference on Machine Learning. 2018.
  [3] Lewis, Adrian S., and Michael L. Overton. "Nonsmooth Optimization via Quasi-Newton
      Methods." Mathematical Programming 141.1-2 (2013): 135-163.
  [4] Liu, Dong C., and Jorge Nocedal. "On the Limited Memory BFGS Method for
      Large Scale Optimization." Mathematical Programming 45.1-3 (1989): 503-528.
  [5] Nocedal, Jorge. "Updating Quasi-Newton Matrices With Limited Storage."
      Mathematics of Computation 35.151 (1980): 773-782.
  [6] Nocedal, Jorge, and Stephen J. Wright. "Numerical Optimization." Springer New York,
      2006.
  [7] Schmidt, Mark. "minFunc: Unconstrained Differentiable Multivariate Optimization
      in Matlab." Software available at http://www.cs.ubc.ca/~schmidtm/Software/minFunc.html
      (2005).
  [8] Schraudolph, Nicol N., Jin Yu, and Simon Günter. "A Stochastic Quasi-Newton
      Method for Online Convex Optimization." Artificial Intelligence and Statistics.
      2007.
  [9] Wang, Xiao, et al. "Stochastic Quasi-Newton Methods for Nonconvex Stochastic
      Optimization." SIAM Journal on Optimization 27.2 (2017): 927-956.
code: def _numel(self):
      if self._numel_cache is None:
          self._numel_cache = reduce(
              lambda total, p: total + p.numel(), self._params, 0
          )
      return self._numel_cache
  
imports: from copy import deepcopy
  from functools import reduce
  import matplotlib.pyplot
  import numpy
  import torch
  from torch.optim import Optimizer
label_desc: Use image node_id 2 for calling the LBFGS obj's underlying member method code with example usage: obj._numel() and returns: self