How to calculate most important features for Random Forest and Logistic Regression with dummies variables?
In general, I'd be really careful about imparting meaning to feature sensitivity (classic correlation is not causation argument), but one major problem is your categories are one hot encoded and spread out, so you need to pick them back up. How you do that somewhat depends on the data and whether you're trying to get importance of an overall category or a label that appears across categories.
I can't write any code because you haven't given any code.
My df had a lot of categorical variables, so I used
pd.get_dummies()
to be able to train my Random Forest and Logistic Regression models. Everything worked fine, then I asked myself: which components affect the models prediction the most? I thought about using PCA, but I have dummies binary variables, so I don't know if it has interpretability due to the number of variables I have being dummies. I also tried using
RF.feature_importances_
but it's the same; I only have thousands of columns with data where each one influences very little, losing data interpretability. Is there any method to calculate the importance of each variable being dummie? I've seen some discussion on stackoverflow about this. Some say PCA can be used, others say it loses interpretability. I do not look for papers that propose methods. In case there is a solution, I would like it to be implemented in python to use it