and momentum respectively.This is in contrast to Sutskever et.
     al. and other frameworks which employ an update of the form
    \begin{aligned}     v_{t+1} & = \mu * v_{t} + \text{lr} *
    g_{t+1}, \\     p_{t+1} & = p_{t} - v_{t+1}. \end{aligned}

 The Nesterov version is analogously modified.Moreover, the
 initial value of the momentum buffer is set to the gradient value
 at the first step. This is in contrast to some other frameworks
 that initialize it to all zeros.

add_param_group(param_group)
  Add a param group to the "Optimizer" s *param_groups*.

  This can be useful when fine tuning a pre-trained network as
  frozen layers can be made trainable and added to the "Optimizer"
  as training progresses.

  Parameters:
     **param_group** (*dict*) -- Specifies what Tensors should be
     optimized along with group specific optimization options.

load_state_dict(state_dict)
  Loads the optimizer state.

  Parameters:
