Optimizer for an RNN using pytorch
The pytorch RNN tutorial uses
for p in net.parameters():
    p.data.add_(p.grad.data, alpha = -learning_rate)

as optimizer. Does anyone know the difference between doing that or doing the classical optimizer.step(), once an optimizer has been defined explicitly? Is there some special consideration one has to take into when training RNNs in regards to the optimizer?

It looks like the example uses a simple gradient descent algorithm to update:

where J is cost.
If the optimizer your using is a simple gradient descent tool, then there is no difference between using optimizer.step() and the code in the example.
I know that's not a super exciting answer to your question, because it depends on how the step() function is written. Check out this page to learn about step() and this page to learn more about torch.optim.
