Pytorch: how to change requires_grad to be true in an OrderedDict
Suppose I have a neural network object from torch.nn, by default the requires_grad is False for its parameters. I want to change it to be True. But the following naive approach fails:
From torch import nn
a = nn.Linear(1, 1)
a.state_dict()[‘weight’].requires_grad = True
print(a.state_dict()[‘weight’].requires_grad)

The result is False. Could anyone explain what the problem is and how to fix it? Thank you! My torch version is 1.7.1.

By default trainable nn objects parameters will have requires_grad=True.
You can verify that by doing:
import torch.nn as nn

layer = nn.Linear(1, 1)

for param in layer.parameters():
    print(param.requires_grad)

# or use
print(layer.weight.requires_grad)
print(layer.bias.requires_grad)

To change requires_grad state:
for param in layer.parameters():
    param.requires_grad = False # or True

