Getting Negative Information Gain (using Entropy as a measure of disorder) I'm creating a Decision Tree and at the very root level itself, I'm getting negative Information Gain. 

As per my knowledge, Information Gain is always &gt; 0....

Any explanation....please....

Please look at the node below.....

    [69+,42-]
     /   \        
    /     \                       
[56+,33-]  [11+,11-]

IG = H([69+, 42-]) - H([56+,33-],[11+,11-]) 

   = {(-69/111 * lg(69/111) -42/111 * lg(42/111)) 
      - 89/111 * ( -56/89*lg(56/89) -33/89*lg(33/89)) 
      - 22/111 * ( -11/22*lg(11/22) -11/22*lg(11/22))}

   = -0.004


It turns out that for every feature, the IG &lt; 0.

What should I do to decide the feature at the root node?

Thanks
It is not possible to have a negative IG, your outcome is negative because of a computational mistake: initially you have 69 positive instances, and after creating 2 children the sum drops to 56+11=67. Set is therefore less pure -> IG is negative.