LSTM regression bias increases when targets go close to 0
I found a solution for the problem: scaling the features and the labels.
By performing min-max scaling over the features and labels, the curves become:

The error is still showing a pattern related to the target value, but the normalized error is now really low...
I think this is a nice example of the benefits of normalizing features...