title: what log value to use for Kullback leibler divergence implementation?
body: <p>I have done a method to implement the Kullback-leibler divergence in java.  I have used   the log with base 2 value and i am not sure whether i have used it right or i should used log base 10 value. I am using this method to measure the divergence between two text units(each of different length).  </p>
  
  <p>My problem is i don't get the desired divergence measure .   </p>
  
  <p>for example for two text units namely => "Free Ringtones" and the second one "Free Ringtones for your Mobile Phone from PremieRingtones.com"  </p>
  
  <p>I should get a divergence of 0.25(as of my project references) but i get a divergence of 2.0 if i use log base2 and 1.38 for log base10.  </p>
  
  <p>Also i am unaware of what value to substitute instead of zero value for demnominator.Plz help in giving clear explaination with some examples if possible and even some links to where i can get details.</p>
  
  <p>This is My code snippet:  </p>
  
  <pre><code>public Double calculateKLD(List&lt;String&gt; values,List&lt;String&gt; value2)   
  {  
  
      Map&lt;String, Integer&gt; map = new HashMap&lt;String, Integer&gt;();  
      Map&lt;String, Integer&gt; map2 = new HashMap&lt;String, Integer&gt;();  
      for (String sequence : values)  
      {  
          if (!map.containsKey(sequence))  
          {  
              map.put(sequence, 0);
          }
          map.put(sequence, map.get(sequence) + 1);
      }
  
      for (String sequence : value2)  
      {  
          if (!map2.containsKey(sequence)) {
              map2.put(sequence, 0);
          }
          map2.put(sequence, map2.get(sequence) + 1);
      }
  
      Double result = 0.0;
      Double frequency2=0.0;
      for (String sequence : map.keySet())  
      {
  
          Double frequency1 = (double) map.get(sequence) / values.size();
          System.out.println("Freuency1 "+frequency1.toString());
          if(map2.containsKey(sequence))
          {
  
              frequency2 = (double) map2.get(sequence) / value2.size();                
          }
          result += frequency1 * (Math.log(frequency1/frequency2) / Math.log(2));         
      }  
      return result/2.4;  
  }    
  </code></pre>
  
  <p>My Input is like this  </p>
  
  <p><strong>First text unit</strong></p>
  
  <pre><code>   list.add("Free");list.add("Ringtones");  
  </code></pre>
  
  <p><strong>Second text unit</strong></p>
  
  <pre><code>      list2.add("Free");list2.add("Ringtones");list2.add("for");list2.add("your");list2.add("Mobiile");list2.add("Phone");list2.add("from");list2.add("PremieRingtones.com");
  </code></pre>
  
  <p>Calling function  </p>
  
  <pre><code>   calculateKLD(list, list2)
  </code></pre>