Explainability ML Methods
You can look at the different evaluations as evaluation of the different stages of the model. 


by evaluating the model's performance on the training dataset, you could assess how and what the model has learned from the data structure. This evaluation is mainly relevant for your research stage and the reporting of your results.
by evaluating the model's performance on the current/test dataset, you would see how the model is behaving and performing on current data the real predictions. This may help you to asses that 


These two may differ in cases where the current data is behaving differently than the data that the model been trained on, and to help you to understand that the model needs to be updated and retrained on the data that it is performing poorly on. 
If you're not monitoring your current data feed (distribution, range of values, the ratio of missings), I would suggest you do both for assessing if your ongoing observed performance is not what you are expecting.