

tolerance_grad (float) -- termination tolerance on first
        order optimality (default: 1e-5).


tolerance_change (float) -- termination tolerance on
    function value/parameter changes (default: 1e-9).


history_size (int) -- update history size (default:
    100).


line_search_fn (str) -- either 'strong_wolfe' or None
    (default: None).




add_param_group(param_group)
  Add a param group to the "Optimizer" s *param_groups*.

  This can be useful when fine tuning a pre-trained network as
  frozen layers can be made trainable and added to the "Optimizer"
  as training progresses.

  Parameters:
     **param_group** (*dict*) -- Specifies what Tensors should be
     optimized along with group specific optimization options.

load_state_dict(state_dict)
  Loads the optimizer state.

  Parameters:
     **state_dict** (*dict*) -- optimizer state. Should be an
