class_docstr: Implements full-batch or deterministic L-BFGS algorithm. Compatible with
  Powell damping. Can be used when evaluating a deterministic function and
  gradient. Wraps the LBFGS optimizer. Performs the two-loop recursion,
  updating, and curvature updating in a single step.
  
  Implemented by: Hao-Jun Michael Shi and Dheevatsa Mudigere
  Last edited 11/15/18.
  
  Warnings:
    . Does not support per-parameter options and parameter groups.
    . All parameters have to be on a single device.
  
  Inputs:
      lr (float): steplength or learning rate (default: 1)
      history_size (int): update history size (default: 10)
      line_search (str): designates line search to use (default: 'Wolfe')
          Options:
              'None': uses steplength designated in algorithm
              'Armijo': uses Armijo backtracking line search
              'Wolfe': uses Armijo-Wolfe bracketing line search
      dtype: data type (default: torch.float)
      debug (bool): debugging mode
docstr: Performs a single optimization step.
  
  Inputs:
      options (dict): contains options for performing line search
  
  General Options:
      'eps' (float): constant for curvature pair rejection or damping (default: 1e-2)
      'damping' (bool): flag for using Powell damping (default: False)
  
  Options for Armijo backtracking line search:
      'closure' (callable): reevaluates model and returns function value
      'current_loss' (tensor): objective value at current iterate (default: F(x_k))
      'gtd' (tensor): inner product g_Ok'd in line search (default: g_Ok'd)
      'eta' (tensor): factor for decreasing steplength > 0 (default: 2)
      'c1' (tensor): sufficient decrease constant in (0, 1) (default: 1e-4)
      'max_ls' (int): maximum number of line search steps permitted (default: 10)
      'interpolate' (bool): flag for using interpolation (default: True)
      'inplace' (bool): flag for inplace operations (default: True)
      'ls_debug' (bool): debugging mode for line search
  
  Options for Wolfe line search:
      'closure' (callable): reevaluates model and returns function value
      'current_loss' (tensor): objective value at current iterate (default: F(x_k))
      'gtd' (tensor): inner product g_Ok'd in line search (default: g_Ok'd)
      'eta' (float): factor for extrapolation (default: 2)
      'c1' (float): sufficient decrease constant in (0, 1) (default: 1e-4)
      'c2' (float): curvature condition constant in (0, 1) (default: 0.9)
      'max_ls' (int): maximum number of line search steps permitted (default: 10)
      'interpolate' (bool): flag for using interpolation (default: True)
      'inplace' (bool): flag for inplace operations (default: True)
      'ls_debug' (bool): debugging mode for line search
  
  Outputs (depends on line search):
    . No line search:
          t (float): steplength
    . Armijo backtracking line search:
          F_new (tensor): loss function at new iterate
          t (tensor): final steplength
          ls_step (int): number of backtracks
          closure_eval (int): number of closure evaluations
          desc_dir (bool): descent direction flag
              True: p_k is descent direction with respect to the line search
              function
              False: p_k is not a descent direction with respect to the line
              search function
          fail (bool): failure flag
              True: line search reached maximum number of iterations, failed
              False: line search succeeded
    . Wolfe line search:
          F_new (tensor): loss function at new iterate
          g_new (tensor): gradient at new iterate
          t (float): final steplength
          ls_step (int): number of backtracks
          closure_eval (int): number of closure evaluations
          grad_eval (int): number of gradient evaluations
          desc_dir (bool): descent direction flag
              True: p_k is descent direction with respect to the line search
              function
              False: p_k is not a descent direction with respect to the line
              search function
          fail (bool): failure flag
              True: line search reached maximum number of iterations, failed
              False: line search succeeded
  
  Notes:
    . If encountering line search failure in the deterministic setting, one
      should try increasing the maximum number of line search steps max_ls.
docstr_tok: ["Performs","a","single","optimization","step",".","Inputs",":","options","(","dict",")",":","contains","options","for","performing","line","search","General","Options",":","'eps","'","(","float",")",":","constant","for","curvature","pair","rejection","or","damping","(","default",":","1e-2",")","'damping","'","(","bool",")",":","flag","for","using","Powell","damping","(","default",":","False",")","Options","for","Armijo","backtracking","line","search",":","'closure","'","(","callable",")",":","reevaluates","model","and","returns","function","value","'current_loss","'","(","tensor",")",":","objective","value","at","current","iterate","(","default",":","F","(","x_k",")",")","'gtd","'","(","tensor",")",":","inner","product","g_Ok","'d","in","line","search","(","default",":","g_Ok","'d",")","'eta","'","(","tensor",")",":","factor","for","decreasing","steplength",">","0","(","default",":","2",")","'c1","'","(","tensor",")",":","sufficient","decrease","constant","in","(","0",",","1",")","(","default",":","1e-4",")","'max_ls","'","(","int",")",":","maximum","number","of","line","search","steps","permitted","(","default",":","10",")","'interpolate","'","(","bool",")",":","flag","for","using","interpolation","(","default",":","True",")","'inplace","'","(","bool",")",":","flag","for","inplace","operations","(","default",":","True",")","'ls_debug","'","(","bool",")",":","debugging","mode","for","line","search","Options","for","Wolfe","line","search",":","'closure","'","(","callable",")",":","reevaluates","model","and","returns","function","value","'current_loss","'","(","tensor",")",":","objective","value","at","current","iterate","(","default",":","F","(","x_k",")",")","'gtd","'","(","tensor",")",":","inner","product","g_Ok","'d","in","line","search","(","default",":","g_Ok","'d",")","'eta","'","(","float",")",":","factor","for","extrapolation","(","default",":","2",")","'c1","'","(","float",")",":","sufficient","decrease","constant","in","(","0",",","1",")","(","default",":","1e-4",")","'c2","'","(","float",")",":","curvature","condition","constant","in","(","0",",","1",")","(","default",":","0.9",")","'max_ls","'","(","int",")",":","maximum","number","of","line","search","steps","permitted","(","default",":","10",")","'interpolate","'","(","bool",")",":","flag","for","using","interpolation","(","default",":","True",")","'inplace","'","(","bool",")",":","flag","for","inplace","operations","(","default",":","True",")","'ls_debug","'","(","bool",")",":","debugging","mode","for","line","search","Outputs","(","depends","on","line","search",")",":",".","No","line","search",":","t","(","float",")",":","steplength",".","Armijo","backtracking","line","search",":","F_new","(","tensor",")",":","loss","function","at","new","iterate","t","(","tensor",")",":","final","steplength","ls_step","(","int",")",":","number","of","backtracks","closure_eval","(","int",")",":","number","of","closure","evaluations","desc_dir","(","bool",")",":","descent","direction","flag","True",":","p_k","is","descent","direction","with","respect","to","the","line","search","function","False",":","p_k","is","not","a","descent","direction","with","respect","to","the","line","search","function","fail","(","bool",")",":","failure","flag","True",":","line","search","reached","maximum","number","of","iterations",",","failed","False",":","line","search","succeeded",".","Wolfe","line","search",":","F_new","(","tensor",")",":","loss","function","at","new","iterate","g_new","(","tensor",")",":","gradient","at","new","iterate","t","(","float",")",":","final","steplength","ls_step","(","int",")",":","number","of","backtracks","closure_eval","(","int",")",":","number","of","closure","evaluations","grad_eval","(","int",")",":","number","of","gradient","evaluations","desc_dir","(","bool",")",":","descent","direction","flag","True",":","p_k","is","descent","direction","with","respect","to","the","line","search","function","False",":","p_k","is","not","a","descent","direction","with","respect","to","the","line","search","function","fail","(","bool",")",":","failure","flag","True",":","line","search","reached","maximum","number","of","iterations",",","failed","False",":","line","search","succeeded","Notes",":",".","If","encountering","line","search","failure","in","the","deterministic","setting",",","one","should","try","increasing","the","maximum","number","of","line","search","steps","max_ls","."]
code: def step(self, options={}):
      """
      Performs a single optimization step.
  
      Inputs:
          options (dict): contains options for performing line search
  
      General Options:
          'eps' (float): constant for curvature pair rejection or damping (default: 1e-2)
          'damping' (bool): flag for using Powell damping (default: False)
  
      Options for Armijo backtracking line search:
          'closure' (callable): reevaluates model and returns function value
          'current_loss' (tensor): objective value at current iterate (default: F(x_k))
          'gtd' (tensor): inner product g_Ok'd in line search (default: g_Ok'd)
          'eta' (tensor): factor for decreasing steplength > 0 (default: 2)
          'c1' (tensor): sufficient decrease constant in (0, 1) (default: 1e-4)
          'max_ls' (int): maximum number of line search steps permitted (default: 10)
          'interpolate' (bool): flag for using interpolation (default: True)
          'inplace' (bool): flag for inplace operations (default: True)
          'ls_debug' (bool): debugging mode for line search
  
      Options for Wolfe line search:
          'closure' (callable): reevaluates model and returns function value
          'current_loss' (tensor): objective value at current iterate (default: F(x_k))
          'gtd' (tensor): inner product g_Ok'd in line search (default: g_Ok'd)
          'eta' (float): factor for extrapolation (default: 2)
          'c1' (float): sufficient decrease constant in (0, 1) (default: 1e-4)
          'c2' (float): curvature condition constant in (0, 1) (default: 0.9)
          'max_ls' (int): maximum number of line search steps permitted (default: 10)
          'interpolate' (bool): flag for using interpolation (default: True)
          'inplace' (bool): flag for inplace operations (default: True)
          'ls_debug' (bool): debugging mode for line search
  
      Outputs (depends on line search):
        . No line search:
              t (float): steplength
        . Armijo backtracking line search:
              F_new (tensor): loss function at new iterate
              t (tensor): final steplength
              ls_step (int): number of backtracks
              closure_eval (int): number of closure evaluations
              desc_dir (bool): descent direction flag
                  True: p_k is descent direction with respect to the line search
                  function
                  False: p_k is not a descent direction with respect to the line
                  search function
              fail (bool): failure flag
                  True: line search reached maximum number of iterations, failed
                  False: line search succeeded
        . Wolfe line search:
              F_new (tensor): loss function at new iterate
              g_new (tensor): gradient at new iterate
              t (float): final steplength
              ls_step (int): number of backtracks
              closure_eval (int): number of closure evaluations
              grad_eval (int): number of gradient evaluations
              desc_dir (bool): descent direction flag
                  True: p_k is descent direction with respect to the line search
                  function
                  False: p_k is not a descent direction with respect to the line
                  search function
              fail (bool): failure flag
                  True: line search reached maximum number of iterations, failed
                  False: line search succeeded
  
      Notes:
        . If encountering line search failure in the deterministic setting, one
          should try increasing the maximum number of line search steps max_ls.
  
      """
  
      # load options for damping and eps
      if "damping" not in options.keys():
          damping = False
      else:
          damping = options["damping"]
  
      if "eps" not in options.keys():
          eps = 1e-2
      else:
          eps = options["eps"]
  
      # gather gradient
      grad = self._gather_flat_grad()
  
      # update curvature if after 1st iteration
      state = self.state["global_state"]
      if state["n_iter"] > 0:
          self.curvature_update(grad, eps, damping)
  
      # compute search direction
      p = self.two_loop_recursion(-grad)
  
      # take step
      return self._step(p, grad, options=options)
  
imports: from copy import deepcopy
  from functools import reduce
  import matplotlib.pyplot
  import numpy
  import torch
  from torch.optim import Optimizer
label_desc: Use image node_id 2 for calling the FullBatchLBFGS obj's underlying member method code with example usage: obj.step(options) and returns: self