How can I get the gradients of two losses in pytorch
I am trying to get the gradients of two losses in the following code snippet but all I get is None (AttributeError: ‘NoneType’ object has no attribute ‘data’)
img = img.to(device)
#img.requires_grad = True
input = model(img)
input_prediction = input.max(1, keepdim=True)[1]

btarget = torch.tensor(2).unsqueeze(0).to(device)
x_prime.requires_grad = True
x_prime_output = model(x_prime)
x_prime_pred =  x_prime_output.max(1, keepdim=True)[1]

l_target_loss = F.nll_loss(x_prime_output, btarget)
# model.zero_grad()
l_target_loss.retain_grad()
l_target_loss.backward(retain_graph = True)
target_grad = l_target_loss.grad.data

l_argmax_loss = F.nll_loss(x_prime_output, input_prediction.squeeze(0))
l_argmax_loss.retain_grad()
l_argmax_loss.backward()
l_argmax_grad = l_argmax_loss.grad.data

when I tried getting target_grad and l_argmax_grad, I get None.
Edit: I am actually trying to get the gradient of the l_target_loss w.r.t to the input x and the gradient of the  l_argmax_loss w.r.t to the input x as well

You need to retain the gradient on that tensor with retain_grad, by default it is not cached in memory:
&gt;&gt;&gt; l_target_loss.retain_grad()
&gt;&gt;&gt; l_target_loss.backward(retain_graph=True)

