Since the development of RNNs , the system was often unable to learn correctly when many recurrent layers were connected together or the input sequence was too long, due to problems such as vanishing and exploding gradients . These were mitigated by using a very specific layer called a Long Short-Term Memory (LSTM). Further optimization led to the Gated Recurrent Unit (GRU), which has been demonstrated to produce similar results at a lower computational cost.