title: How to find feature importance in a Weka-built decision tree
body: <p>I used Weka to successfully build a J48 (C4.5) decision tree. I would now like to evaluate how effective or important my features are.</p>
  
  <p>One obvious way is to loop through all the features, remove one at a time, and re-run classification tests each time to see which feature has the largest drop in classification accuracy. However, this may hide co-dependencies between features.</p>
  
  <p>However, I am thinking of another approach based on understanding the C4.5 algorithm. Since each split in the tree is based on a maximum information gain decision, a split on a feature closer to the root of the tree must mean that the feature had more information gain than a split with a different feature lower in the tree. So for a given feature F that occurs in several splits within the tree, I can calculate F's average distance away from the root. I can then rank all the features by average distance, with the lowest average being the most valuable feature. Would this be a correct approach?</p>