
differentiable (bool, optional) -- whether autograd
        should occur through the optimizer step in training.
        Otherwise, the step() function runs in a torch.no_grad()
        context. Setting to True can impair performance, so leave it
        False if you don't intend to run autograd through this
        instance (default: False)

-[ Example ]-



optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
optimizer.zero_grad()
loss_fn(model(input), target).backward()
optimizer.step()



Note:
 The implementation of SGD with Momentum/Nesterov subtly differs
 from Sutskever et. al. and implementations in some other
 frameworks.Considering the specific case of Momentum, the update
 can be written as

    \begin{aligned}     v_{t+1} & = \mu * v_{t} + g_{t+1}, \\
    p_{t+1} & = p_{t} - \text{lr} * v_{t+1}, \end{aligned}

 where p, g, v and \mu denote the parameters, gradient, velocity,
