Forward and backward process in pyTorch When I write a network, do I have to write the whole forward property in nn.Module.forward()? I mean if I do some operations outside the net, does grad correctly flow?

For example, I have two networks, in which the output of one is the input of the other(net1 -&gt; midresults -&gt; net2). 

If I do some operations on midresults(net1 -&gt; midresults -&gt; operations -&gt; net2), can (net1+net2) be trained end to end?
As long as your operations are all compatible with pytorch tensors and Autograd then yes your network will be trained end-to-end.

A good rule of thumb is to ask yourself : I'm I using tensors and pytorch operators end-to-end? If yes (and unless stated otherwise in the docs), you should be pretty safe.