How do we determine what is learnt by a ML model? I am trying to investigate/justify the output of a random forest regression model for a financial problem, where justification of the output is also important. In that context, for the random forest regression model that I built, I'm using the tree interpreter for justification of the output using the contributions. But is there any other way I can do this better? Is this the best approach? 

The idea is to be able to explain the output of the random forest regression model to a layman supported with technical points.
Shap values is your friend.

model = RandomForestRegressor(max_depth=6, random_state=0, n_estimators=10)
model.fit(X_train, Y_train)
import shap
shap_values = shap.TreeExplainer(model).shap_values(X_train)
shap.summary_plot(shap_values, X_train, plot_type="bar")


There are several other techniques with their own drawbacks. Some are:


Permutation Importance
Partial Dependence plots
LIME
Global Surrogate Models
etc.. You can add to this list.