intermediate = f(input[i])
       result += g(intermediate)
   output = h(result)
   return output
Here, "intermediate" remains live even while "h" is executing, because
its scope extrudes past the end of the loop.  To free it earlier, you
should "del intermediate" when you are done with it.
Avoid running RNNs on sequences that are too large. The amount of
memory required to backpropagate through an RNN scales linearly with
the length of the RNN input; thus, you will run out of memory if you
try to feed an RNN a sequence that is too long.
The technical term for this phenomenon is backpropagation through
time, and there are plenty of references for how to implement
truncated BPTT, including in the word language model example;
truncation is handled by the "repackage" function as described in this
forum post.
Don't use linear layers that are too large. A linear layer
"nn.Linear(m, n)" uses O(nm) memory: that is to say, the memory