One of the significant challenges associated with RNNs is their inability to handle long-term dependencies when the unfolded computational graph becomes extremely deep. Specifically, when the forward pass of an RNN involves repeatedly multiplying a weight matrix 𝑾 to the state 𝒉 at each time step, as depicted in Figure , the gradient may either vanish or explode, rendering the RNN difficult to train. The vanishing gradient problem occurs when the gradient becomes extremely small, making it difficult for the network to learn from past information. On the other hand, the exploding gradient problem occurs when the gradient becomes exceedingly large, making the network learning unstable. These issues are particularly problematic in deep architectures and can severely limit the performance of RNNs.