Rescaling time-series data with very spiky pattern for training data in LSTM network
First, if your data has a minimum of 0 and maximum of 5000, 1000 will get rescaled to .2 and 5000 will get rescaled to 1. So it's not a .001 difference as you suggest.
If you just used a regular loss function (e.g. mean squared error), I'm not sure you'd be able to achieve good predictions. In the literature this type of data might be called &quot;sharp&quot;. There are some special loss functions such as DILATE (link) or soft-DTW (link) which are specially designed for time series like the one you show.