How do we determine what is learnt by a ML model?
Shap values is your friend.

model = RandomForestRegressor(max_depth=6, random_state=0, n_estimators=10)
model.fit(X_train, Y_train)
import shap
shap_values = shap.TreeExplainer(model).shap_values(X_train)
shap.summary_plot(shap_values, X_train, plot_type="bar")


There are several other techniques with their own drawbacks. Some are:


Permutation Importance
Partial Dependence plots
LIME
Global Surrogate Models
etc.. You can add to this list.