Pytorch - Trick AutoGrad into thinking another output is the final outcome
The scenario:

I have a simple torch CNN network that predicts if a given image input is a dog or a cat.

After getting the output of the neural network, I need to apply a modifier of X to each prediction. For example, if the neural network return [0.6, 0.4], and I want to apply a modifier of [0.05, -0.03], I need the result to be [0.65, 0.37].


The desired result:

I would like AutoGrad to think the final output is [0.65, 0.37]. That is, AutoGrad shouldn't consider the modifier addition at all. In fact, It needs to be tricked into thinking the last operation result is [0.65, 0.37] instead of [0.6, 0.4], and apply the backpropagation with that in mind.

Here's what I need to know:

How am I suppose to do this? I know Torch records every single operation and compute a dynamic graph accordingly. I don't want this operation to be recorded at all, and I can't use the torch.no_grad() wrapper, because I need to be able to do the backpropagation after.


Edit 1
@trsvchn Here's what happen when I do your method. Is this expected? Will autograd use the &quot;T&quot; value instead of the &quot;data&quot; value when doing the backpropagation?


Use torch.no_grad context manager to deactivate autograd, for example this operation is not going to be recorded, but you should use in-place operation, otherwise you will add another tensor without grad_fn and break the graph. In your case:
out = model(inputs)  # [0.6, 0.4]

with torch.no_grad():
    out.add_(torch.tensor([0.05, -0.03]))  # inplace add_ op

