How to evaluate performance of a time series model?
The best summary on evaluating time series forecast is probably explained in detail on Rob Hyndman's site. I typically use the mean absolute percentage error which is baked in Keras. However, what I found in a different setting is that the MAPE prevents the neural network from converging if combined with the Adam optmization. I had much better success with the rooted mean square error (RMSE). Since you have poor experience with that maybe you could use the symmetric MAPE.