Regression and Neural networks I'm trying to restore this function:

$$ F(x) = x*sin(\alpha x)+b; \space\space \alpha,b \in (-20,20) $$

My NN model(with Keras) is:  

1 layer: GRU, 9 neurons, selu activation
2 layer: GRU, 3 neurons, selu activation
1 layer: GRU, 7 neurons, selu activation
1 layer: Dense, 1 neuron, linear activation
kernel init he_normal for all layers  

For the training dataset I've generated function values $f(x), x \in -5..5$ (e.g. linspace(-5,5,500) and random integer values for $\alpha$ and $b$, 250 times. 

Then I've selected $f[i-1]$ and $f[i]$ (previous steps) and for this "xs" the output is $f[i+1]$. So the training dataset is like:  

X:
   first row: previous previous $f$, $\alpha$, $b$
   second row: previous $f$, $\alpha$, $b$  

Y: 
   current $f$ value  

After training over 200 epochs, mae on validation data was 0.1851.

Now if I try to predict new data it seems normal, but when I try to predict new values using previously predicted by model points it falls down and it doesn't looks like sine function at all.

What am I doing wrong?
The problem is that your error is accumulating and diverging. In other words, a small error in the first prediction is leading to a larger error in your second prediction, which is leading to an even larger error in your third prediction, and so on.

With that said, this LSTM example seems to do well predicting multiple future values for a sine wave.