title: Calculating Entropy
body: <p>I've tried for several hours to calculate the Entropy and I know I'm missing something. Hopefully someone here can give me an idea!</p>
  
  <p>EDIT: I think my formula is wrong!</p>
  
  <p><strong>CODE:</strong></p>
  
  <pre><code> info &lt;- function(CLASS.FREQ){
        freq.class &lt;- CLASS.FREQ
        info &lt;- 0
        for(i in 1:length(freq.class)){
          if(freq.class[[i]] != 0){ # zero check in class
            entropy &lt;- -sum(freq.class[[i]] * log2(freq.class[[i]]))  #I calculate the entropy for each class i here
          }else{ 
            entropy &lt;- 0
          } 
          info &lt;- info + entropy # sum up entropy from all classes
        }
        return(info)
      }
  </code></pre>
  
  <p>I hope my post is clear, since it's the first time I actually post here.</p>
  
  <p><strong>This is my dataset:</strong></p>
  
  <pre><code>buys &lt;- c("no", "no", "yes", "yes", "yes", "no", "yes", "no", "yes", "yes", "yes", "yes", "yes", "no")
  
  credit &lt;- c("fair", "excellent", "fair", "fair", "fair", "excellent", "excellent", "fair", "fair", "fair", "excellent", "excellent", "fair", "excellent")
  
  student &lt;- c("no", "no", "no","no", "yes", "yes", "yes", "no", "yes", "yes", "yes", "no", "yes", "no")
  
  income &lt;- c("high", "high", "high", "medium", "low", "low", "low", "medium", "low", "medium", "medium", "medium", "high", "medium")
  
  age &lt;- c(25, 27, 35, 41, 48, 42, 36, 29, 26, 45, 23, 33, 37, 44) # we change the age from categorical to numeric
  </code></pre>