title: Why do we select Entropy Gain as criteria in Decision Tree learning instead of decrease in error rates as a criteria?
body: <p>I've been following the <a href="http://www.cs.cmu.edu/~ninamf/courses/601sp15/lectures.shtml" rel="nofollow noreferrer">ML course by Tom Mitchel</a> and in Decision Tree (DT) Learning, the <strong>Entropy Gain</strong> is chosen as ruling criterion for the choice of a feature/parameter <code>x_i</code> as child of another feature in DT top-down growth.</p>
  
  <p>Always our goal of selecting a DT is to avoid <strong>overfitting</strong> by minimizing the error rates; <strong>then why don't we use error rate as a ruling criteria for feature/parameter selection in top-down growth of the tree.</strong></p>
  
  <p>Feature vector for Input data: <code>X = &lt; x_1, x_2......x_n &gt;</code></p>