title: Information Gain Calculation for 3 Classes
body: <p>I have a problem that requires us to use information gain for feature selection but the target class has 3 classes (positive, negative, and neutral) instead of two. As such, I am a little confused about how the calculation for info would work.</p>
  <p>For 2 class problems, I know that info(p,n) = 0 if all the objects are in 1 class and there are none in the other class. Would this be the same for 3-class problems? How about if 2 classes have objects and the third class does not?</p>
  <p>For example, if I have an attribute split as follows, would the following info(p,n,neu) calculations be correct?</p>
  <p>The formula I am using for info is -p/p+n+neulog2(p/p+n+neu)-n/p+n+neulog2(n/p+n+neu)-neu/p+n+neulog2(neu/p+n+neu)</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>Dept</th>
  <th>Positive</th>
  <th>Negative</th>
  <th>Neutral</th>
  <th>Info(p,n,neu)</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>X</td>
  <td>40</td>
  <td>34</td>
  <td>0</td>
  <td>0</td>
  </tr>
  <tr>
  <td>Y</td>
  <td>60</td>
  <td>70</td>
  <td>70</td>
  <td>0.3969</td>
  </tr>
  <tr>
  <td>Z</td>
  <td>34</td>
  <td>0</td>
  <td>36</td>
  <td>0</td>
  </tr>
  <tr>
  <td>A</td>
  <td>68</td>
  <td>83</td>
  <td>0</td>
  <td>0</td>
  </tr>
  <tr>
  <td>B</td>
  <td>0</td>
  <td>50</td>
  <td>0</td>
  <td>0</td>
  </tr>
  </tbody>
  </table>
  </div>