Pytorch guide not using optim to train
I am currently working through a guide on the Pytroch website here: https://pytorch.org/tutorials/intermediate/char_rnn_classification_tutorial.html
I have done pytorch projects before and they have always made use of an optimizer. This guide instead uses the code here:
    # Add parameters' gradients to their values, multiplied by learning rate
    for p in rnn.parameters():
        p.data.add_(p.grad.data, alpha=-learning_rate)

I was confused by this and wanted to know why this works. Additionally I tried to rewrite the code using an optimizer and it was unable to learn. It is using a recurrent neural network which may be the reason, but I am unsure why. Thanks!

Why do you expect it to not work? Basically what it is doing is manually implementing an optimizer. p.data is the stored value of the parameter. It also provides an internal function add_ that calculates +=. Once loss.backward() is called, pytorch also calculates and stores the gradient. It is simply taking the the gradient value from the backward pass and updating the parameters to perform gradient descent. There is no reason an optimizer shouldn't work here either, but I can't help with that unless you give more info.
