Big and complex datasets might require more flexibility than what single-layer RNNs offer. For a better sequence representation, we might as well stack multiple recurrent layers hypothesizing that higher layers capture longer-term dynamics and lower layers shorter-term. For a K -layer RNN, the k th layer at time step t is H (k ) t and it depends on the input H (k-1) t and the hidden state