Gradient Matrix (NxWxEPOCH) using Pytorch
I'm trying to create a matrix of gradient with the gradient of each observation by parameters and EPOCH. If my model has 100 obs, 1000 params and 10 Epoch, my matrix should be (100,1000,10).
The problem is that I'm not able to get those gradient. The parameters and the observation are set at required_gradient=True.
I've tried to run this after each observation pass thru the net:
 for p in net.parameters():
     paramgradlist.append(p.grad)

But the gradient stays the same of each params stays the same for all observations.
Thank you

You are not copying your data and instead of storing a reference to the gradients. In the end, this means all your observations will be the same (i.e. the gradients' final value).
Instead, you could clone the gradients before appending them to the list:
for p in net.parameters():
    paramgradlist.append(p.grad.clone())

