add a nn.Parameter tensor to the optim
let's say I have a network with a parameter tensor:
self.inital_value = nn.Parameter(torch.randn(1,96,4,4))

and two layers:
self.rgb_layers = nn.ModuleList([nn.Conv2d(96, 3, 1, 1, 0)]*2)

so the forward function is as follows:
v = torch.clone(self.inital_value)
for i in range(2):
    v = self.rgb_layers[i](v)
return v

now I define an optimiser. if I want to train only the first layer I can do:
opt = optim.RMSprop(model.rgb_layers[0].parameters(),lr = lr)

but if I want to train only the tensor self.inital_value and I do the same I get this error:
'Parameter' object has no attribute 'parameters'

how can I bypass this limitation?

Well, it takes a sequence of Parameters. And you have one Parameter, called self.initial_value.
So how about [self.initial_value] ?
