Pytorch gradient descent keeps sending me NaNs mean squared errors
I am trying to apply, within the framework of a course, a gradient descent to estimate a linear model. My code is the following :
model = torch.nn.Linear(1,1)
myModel = model(X)
ds = torch.utils.data.TensorDataset(X, Y)
dl = torch.utils.data.DataLoader(ds)
optimiser = torch.optim.SGD(model.parameters(), lr=0.01)
loss = torch.nn.functional.mse_loss

for epoch in range(100):
    for (Xb, yb) in dl:
        yb_pred = model(Xb)
        c_loss = loss(yb_pred, yb)
        print(c_loss)
        optimiser.zero_grad()
        c_loss.backward()
        optimiser.step()

Yet it keeps printing NaNs, which I do not understand. Have I done a mistake in the implementation ? I have the following output (x numerous times) :

tensor(nan, grad_fn=&lt;MseLossBackward0&gt;)


There is nothing wrong with your code but Nan values can be explained by gradients exploding depending on the data X and Y. You can try with a lower learning rate (1e-3 or 1e-4).
For instance if you test with this toy linear example:
X = torch.randn(100, 1)
Y = X * 2 + 3

The loss will converge to 0 quickly.
