how is stacked rnn (num layers > 1) implemented on pytorch?
The GRU layer in pytorch takes in a parameter called num_layers, where you can stack RNNs. However, it is unclear how exactly the subsequent RNNs use the outputs of the previous layer.
According to the documentation:
Number of recurrent layers. E.g., setting num_layers=2 would mean stacking two GRUs together to form a stacked GRU, with the second GRU taking in outputs of the first GRU and computing the final results.
Does this mean that the output of the final cell of the first layer of the GRU is fed as input to the next layer? Or does it mean the outputs of each cell (at each timestep) is fed as an input to the cell at the same timestep of the next layer?


Does this mean that the output of the final cell of the first layer of the GRU is fed as input to the next layer? Or does it mean the outputs of each cell (at each timestep) is fed as an input to the cell at the same timestep of the next layer?

The latter. Each time step's output from the first layer is used as input for the same time step of the second layer.
This figure from a Keras tutorial shows how multilayer RNNs are structured:

