How to scale exponential data for a regression problem? I understand that I should be scaling features between (0, 1) before feeding them into a neural network. However, what happens if future data could be larger than my current training data? For instance, if I am training a RNN on time-series data to perform demand forecasting, the product I am forecasting may be in a growth stage that will yield higher demand numbers in the future months than the past months. Is there a way to normalize data but still allow for larger values in the future?
Yes, there is. Instead of Min-Max scaling, that shrinks any distribution in the [0, 1] interval, you can Scale the variables (in statistics, they are called Z-scores). The formula is:

(x - mean(x)) / stdev(x)


This will zero-center the distribution, and scale all the variables to a standard deviation = 1.

I strongly prefer this technique to Min-Max Scaling, since the latter is too sensitive to outlier observations and generates problems unseen, out-of-scale datapoints.

Additionally, if your analysis allows for that, consider taking the log() of a data distribution that grows exponentially. That would coerce it to a more linear growth. (I'm not sure this is what you need, it's just a possibility.)