How is error back-propagated in a multi-layer RNN
As you can see here, derivatives will be propagated by the chain rule although they are stacked. Actually, there will be two main paths. The first one will be backpropagation through time and the next one will be the backpropagation from the output of each unrolled cell which can directly be connected to the output or can be connected to the stacked unrolled cells. Also, take a look at here.