Pytorch - Are gradients transferred on creation of new Variables?
I have the following code:

A = Tensor of [186,3]


If I create a new empty tensor as follows:

tempTens = torch.tensor(np.zeros((186,3)), requires_grad = True).cuda()


And I apply some operations on a block of A and output it into tempTens, which I use totally for further computation, say like this:

tempTens[20,:] = SomeMatrix * A[20,:] 


Will the gradients actually be transferred correctly, lets say I am having a cost function that optimizes for the output of tempTens to some ground truth

In this case, tempTens[20,:] = SomeMatrix * A[20,:] is an in-place operation with respect to tempTens, which is generally not guaranteed to work with autograd. However, if you create a new variable by applying an operation like concatenation

output = torch.cat([SomeMatrix * A[20, :], torch.zeros(163, 3, device='cuda')], dim=0)


you will get the same result in terms of math (a matrix with first 20 rows from SomeMatrix * A[20, :] and the following 166 rows of 0s), but this will work properly with autograd. This is, generally speaking, the right way to approach this kind of problems.
