What metrics determine the quality of the model?
Unscaled and scaled r2's are not highly correlated (0.31 AAMOF). Which one would best describe the accuracy of the model on unseen data?


I don't think this is a matter of which will describe the generalization error better, because both of them are describing the same thing, just on different scales. So, the advice would be to use the accuracy metric consistent with the metric that will be used for predictions on unseen data.


  Why isn't the unscaled r2 the same as the scaled r2?


This is because MSE is scale dependent.


  The model r2 is not the same as any of the validation r2's during training (val_r2_keras). Shouldn't the trained model r2 be the same as the one reported during the training?


Why do you think so?

They are different because the datasets for training and for validation are different.