"residual error" of LSTM during backprop vs usual "error"
Residual errors are the errors that remain after a model has tried fitting to some data. It is the error which resides.

People use that term, along with just error or residuals interchangeably, but after a model has been tested, it just means how much of the data cannot be explained by the model.

The letter $\epsilon$ is commonly used to explain stochastic noise inherent in (co)-variates of a model, i.e. noise.error that we cannot explain with the given data.