sklearn's feature importances_
It means those words are "strongly associated" with one of the responses, in your case probably illegal(1). Depending on your classifier, the exact technical definition of strongly associated will vary. It could be the joint probability of the word and response, P(X='theft', Y='illegal'), or it could be the conditional probabilityP(X='theft' | Y='illegal').
Intuitively, whenever these terms appear in a document, the probability of that document belonging to the illegal category is increased.
I am just curious on the interpretation of sklearn's feature_importances_ attribute. I know that the features with highests coefficients are the features that would highly predict the outcome. My question is - Are these the features strongly predictive to return a 1 (or yes) or not necessarily? (Supervised Learning - Binary response - yes(1) or no(0)).
For example, after building the predictive model, I found out that these words are the top features - insider-trading, theft, embezzlement, investment. The response is 'illegal'(1) or 'legal'(0).
Does it mean that when a certain text has those words, there's a huge chance it's illegal or not necessarily? And, it just simply means that the value of these words would lead to a strong prediction (either illegal or legal). Appreciate any answer to such.