Feature importance over a subset of instance space instead of an entire instance space I'm really curious if anyone has faced this problem before, or is it even widely studied at all. 

Imagine we have a feature that isn't important (based on many widely available and textbook feature selection methods) over the entire instance space of training data. However, if we try to  divide the instance space into certain subsets, can the feature become important over that subset, and are there any methods available for arbitrary datasets?

When I say 'dividing the instance space', I mean for example if I have 3 features $X_0, X_1, X_2$ with a binary label $Y$, a subset of instance space will be a region where, for example, when $X_0 &gt;5, X_1&gt;5$. In particular, could it be possible that $X_2$ isn't significant over the entire instance space, but is significant over the restricted space mentioned above. If it's possible (my intuition says so), are there any methods in current literature that allow me to find such a space (if it exists)?

If anyone has any links/papers/resources which can direct me to a similar study, I would gladly accept it as well.
This is a common occurrence in practice. For example, in the bioinformatics space, a protein can change a phenotype significantly over a very limited range, but outside that range has little to no effect.

If you want to where the 'range of highest effect' falls I recommend partial dependence plots which are means of ICE plots. It offers an advantage over decision trees in that you won't overfit as easily to your training dataset and can be applied to high dimensional data. Here's an example of the plot. Read more here. The following packages have some implementation of the above:


mlinsights - https://ml-insights.readthedocs.io/en/latest/
https://media.readthedocs.org/pdf/pdpbox/latest/pdpbox.pdf


Here's a decent example from the above:


If you only want to know if something is important, not where it is important. You can use something called permutation importance or Mean Decrease in Accuracy outlined very briefly in a seminal random forest paper, page 20-21: https://www.stat.berkeley.edu/~breiman/randomforest2001.pdf

Look into the eli5 package for easy calculation.