Comparing Parameter Importance Across Models I have a dataset with 20 features(columns that is). I create a few models pairs with a subset of these parameters.

For example: If I have 6 columns (named A, B, C, D, E, F) with 10k lines of data, one of model in the pair will have (A, B, C) with all 10k lines of data and the other (A, B, C) with maybe say 6k lines of data. Yet another pair will have the parameters (B, C, E), however, the number of lines is always fixed at 10k and 6k.

I am limited to using a sci-kit learn's decision tree and this cannot be changed. I use the following function to calculate my features:

model = clf_gini.best_estimator_
print(model.feature_importances_)


My question is: **Is there any formal, programmatic way to compare feature importance across model pairs using sci-kit, other than intuition? Also, does it make sense to compare to models trained on some similar some different parameters? **

Please let me know if the question is ambiguous, I will clarify.
This may not be the answer you are hoping for (I can't leave a comment because my reputation is too low) but I believe feature importance in Sci-Kit Learn is derived from the average depth at which each feature appears. I don't know of any built-in functions in Sci-Kit Learn but perhaps you could scale the feature importance using the depth of the tree and the number of features?

This article may help with developing a comparative measure of feature importance: https://medium.com/the-artificial-impostor/feature-importance-measures-for-tree-models-part-i-47f187c1a2c3.