PyTorch shows the error " 'NoneType' object has no attribute 'zero_' " when calling the zero_ method
Can someone please answer why my code is showing an error.
Thanks in advance.
Code:
import torch
torch.manual_seed(0)

a = torch.rand((1, 3), requires_grad = True)
w1 = torch.rand((3, 3), requires_grad = True)
w2 = torch.rand((3, 1), requires_grad = True)

d = torch.matmul(torch.matmul(a, w1), w2)

L = (10 - d)

L.backward()

w1 = w1 - w1.grad*0.001
w1.grad.zero_()

Error:
AttributeError
'NoneType' object has no attribute 'zero_'

The line
w1 = w1 - w1.grad*0.001

is reassigning w1, so afterwards w1 no longer refers to the same tensor it did before. To maintain all the internal state of w1 (e.g. the .grad member) you must update w1 in place. Since this is a leaf tensor we also need to disable construction of the computation graph.
with torch.no_grad():
    w1.sub_(w1.grad * 0.001)

