Pytorch loss.backward() gives none grad for parameters of Rx, Ry Gate
I'm trying to train parameters params by performing linear Transformation on an input tensor x by matrix multiplying Rx to input followed by Ry matrix to their result. (each matrix Rx and Ry have a parameter params[i] each that define the matrix).
then I calculate loss by mse of y and the predicted output. when I do loss.backward()
Im getting params.grad  as None.
import torch

def get_device(gpu_no):
    if torch.cuda.is_available():
        return torch.device('cuda', gpu_no)
    else:
        return torch.device('cpu')


device = get_device(0)

params = torch.tensor(([[0.011], [0.012]]), requires_grad=True).to(device).to(torch.cfloat)


x_gate = torch.tensor([[1., 0.], [0., 1.]]).to(device)
y_gate = torch.tensor(([[0, -1j], [1j, 0]])).to(device)


def rx(theta):
    # co = torch.cos(theta / 2)
    # si = torch.sin(theta / 2)
    # Rx_gate = torch.stack([torch.cat([co, -si], dim=-1),
    #                       torch.cat([-si, co], dim=-1)], dim=-2).squeeze(0).to(device).to(torch.cfloat).requires_grad_()
    # Rx_gate = torch.exp(-1j * (theta / 2) * x_gate).to(device).to(torch.cfloat).requires_grad_()
    Rx_gate = torch.tensor(([[torch.cos(theta/2), -torch.sin(theta/2)],
                             [-torch.sin(theta/2), torch.cos(theta/2)]]), requires_grad=True).to(device).to(torch.cfloat)

    return Rx_gate


def ry(theta):
    # co = torch.cos(theta / 2)
    # si = torch.sin(theta / 2)
    # Ry_gate = torch.stack([torch.cat([co, -si]),
    #                             torch.cat([si, co])], dim=-2).squeeze(0).to(device).to(torch.cfloat).requires_grad_()
    # Ry_gate = torch.exp(-1j * (theta / 2) * y_gate).to(device).to(torch.cfloat).requires_grad_()
    Ry_gate = torch.tensor(([[torch.cos(theta / 2), -torch.sin(theta / 2)],
                        [torch.sin(theta / 2), torch.cos(theta / 2)]]), requires_grad=True).to(device).to(torch.cfloat)

    return Ry_gate


x = torch.tensor([1., 0.]).to(device).to(torch.cfloat)
y = torch.tensor([0., 1.]).to(device).to(torch.cfloat)


def pred(params):
    out = rx(params[0]) @ x
    out = ry(params[1]) @ out
    return out

print(&quot;params        :&quot;, params)
print(&quot;prediction    :&quot;, pred(params))

loss = torch.pow((y - pred(params)), 2).sum()
print(&quot;loss          :&quot;, loss)

loss.backward()
print(&quot;loss grad     :&quot;, loss.grad)
print(&quot;params grad   :&quot;, params.grad)

my output is
params        : tensor([[0.0110+0.j],
        [0.0120+0.j]], device='cuda:0', grad_fn=&lt;ToCopyBackward0&gt;)
prediction    : tensor([1.0000e+00+0.j, 5.0000e-04+0.j], device='cuda:0',
       grad_fn=&lt;MvBackward0&gt;)
loss          : tensor(1.9990+1.7485e-07j, device='cuda:0', grad_fn=&lt;SumBackward0&gt;)
loss grad     : None
params grad   : None


why grad is none even though params has grad_fn=&lt;ToCopyBackward0&gt;.
Also i get this warning:
 UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at  aten\src\ATen/core/TensorBody.h:417.)

return self._grad

Good observation, you indeed have a correct backpropagation of the gradient through the gradient. So why are you getting none when accessing your parameter?

The reason why you can't access the gradient of this parameter is that only leaf tensors have their gradient cached in memory. Here, since params is a copy of a leaf-tensor (you called to twice on it which made that happen), it will not be considered a gradient of the computation graph.
In order to get access to the gradient of that parameter at runtime, you can force the engine to cache and make it accessible outside with a simple call to retain_grad as suggested by the warning message.
params.retain_grad()

