

differentiable (bool, optional) -- whether autograd
        should occur through the optimizer step in training.
        Otherwise, the step() function runs in a torch.no_grad()
        context. Setting to True can impair performance, so leave it
        False if you don't intend to run autograd through this
        instance (default: False)

fused (bool, optional) -- whether the fused
    implementation (CUDA only) is used. Currently,
    torch.float64, torch.float32, torch.float16, and
    torch.bfloat16 are supported. Since the fused implementation
    is usually significantly faster than the for-loop
    implementation, we try to use it whenever possible (all
    parameters are on CUDA and are of a supported type). Else, we
    continue with the for-loop implementation. (default: None)



add_param_group(param_group)
  Add a param group to the "Optimizer" s *param_groups*.
