Gradient is none in pytorch when it shouldn't
I am trying to get/trace the gradient of a variable using pytorch, where I have that variable, pass it to a first function that looks for some minimum value of some other variable, then the output of the first function is inputted to a second function, and the whole thing repeats multiple times. 

Here is my code:

import torch

def myFirstFunction(parameter_current_here):
    optimalValue = 100000000000000
    Optimal = 100000000000000
    for j in range(2, 10):
        i = torch.ones(1, requires_grad=True)*j
        with torch.enable_grad():
            optimalValueNow = i*parameter_current_here.sum()
        if (optimalValueNow &lt; optimalValue):
            optimalValue = optimalValueNow
            Optimal = i
    return optimalValue, Optimal

def mySecondFunction(Current):
    with torch.enable_grad():
        y = (20*Current)/2 + (Current**2)/10
    return y

counter = 0
while counter &lt; 5:
    parameter_current = torch.randn(2, 2, requires_grad=True)

    outputMyFirstFunction = myFirstFunction(parameter_current)
    outputmySecondFunction = mySecondFunction(outputMyFirstFunction[1])
    outputmySecondFunction.backward()

    print("outputMyFirstFunction after backward:",
               outputMyFirstFunction)
    print("outputmySecondFunction after backward:",
               outputmySecondFunction)
    print("parameter_current Gradient after backward:",
               parameter_current.grad)

    counter = counter + 1


The parameter_current.grad is none for all iterations when it obviously shouldn't be none. What am I doing wrong? And how can I fix it?

Your help on this would be highly appreciated. Thanks a lot!

Aly

I have a similar experience with this. Reference: https://pytorch.org/docs/stable/tensors.html


For Tensors that have requires_grad which is True, they will be leaf Tensors if they were created by the user. This means that they are not the result of an operation and so grad_fn is None.
Only leaf Tensors will have their grad populated during a call to backward(). To get grad populated for non-leaf Tensors, you can use retain_grad().
Example:


    &gt;&gt;&gt; a = torch.tensor([[1,1],[2,2]], dtype=torch.float, requires_grad=True)
    &gt;&gt;&gt; a.is_leaf
    True
    &gt;&gt;&gt; b = a * a
    &gt;&gt;&gt; b.is_leaf
    False
    &gt;&gt;&gt; c = b.mean()
    &gt;&gt;&gt; c.backward()
    &gt;&gt;&gt; print(c.grad)
    None
    &gt;&gt;&gt; print(b.grad)
    None
    &gt;&gt;&gt; print(a.grad)
    tensor([[0.5000, 0.5000],
            [1.0000, 1.0000]])
    &gt;&gt;&gt; b = a * a
    &gt;&gt;&gt; c = b.mean()
    &gt;&gt;&gt; b.retain_grad()
    &gt;&gt;&gt; c.retain_grad()
    &gt;&gt;&gt; c.backward()
    &gt;&gt;&gt; print(a.grad)
    tensor([[1., 1.],
            [2., 2.]])
    &gt;&gt;&gt; print(b.grad)
    tensor([[0.2500, 0.2500],
            [0.2500, 0.2500]])
    &gt;&gt;&gt; print(c.grad)
    tensor(1.)

