title: Kullback-Leibler distance between 2 samples
body: <p>I am working with data that belong to 2 groups, A and B. I am trying to find the variable that shows the biggest difference between the 2 populations and I thought the Kullback-Leibler distance would be a good measure for that. Here's a sample that represent my data</p>
  
  <pre><code>df1 &lt;- structure(list(Var1 = c(2L, 3L, 5L, 7L, 2L, 1L, 0L, 0L, 0L, 1L, 
  3L, 4L), VarA = c(0.56, 0.43, 0.25, 0.12, 0.78, 0.55, 0.35, 0.36, 
  0.3, 0.41, 0.43, 0.5), VarT = c(10L, 11L, 15L, 12L, 8L, 7L, 7L, 
  7L, 6L, 5L, 1L, 2L), Var3 = c(152L, 187L, 149L, 132L, 132L, 178L, 
  240L, 205L, 137L, 125L, 124L, 56L), group = structure(c(1L, 1L, 
  1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "A", class = "factor")), .Names = c("Var1", 
  "VarA", "VarT", "Var3", "group"), class = "data.frame", row.names = c(NA, 
  -12L))
  
  df2 &lt;- structure(list(Var1 = c(5L, 8L, 7L, 4L, 5L, 2L, 1L, 2L, 6L, 5L
  ), VarA = c(0.24, 0.76, 0.43, 0, 0.52, 0.63, 0.46, 0.64, 0.55, 
  0.78), VarT = c(10L, 8L, 9L, 5L, 11L, 14L, 12L, 1L, 7L, 7L), 
      Var3 = c(205L, 120L, 531L, 203L, 215L, 224L, 211L, 212L, 
      134L, 222L), group = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 
      1L, 1L, 1L, 1L), .Label = "B", class = "factor")), .Names = c("Var1", 
  "VarA", "VarT", "Var3", "group"), class = "data.frame", row.names = c(NA, 
  -10L))
  </code></pre>
  
  <p>I am thinking of applying the Kullback-Leibler distance test in a for loop for similar columns to see the variable that shows the largest distance between the same variable in the 2 groups.</p>
  
  <p>To start with, I have tried to run this command from the <code>FNN</code> package</p>
  
  <pre><code>require(FNN)
  X &lt;- df1[,2]
  Y &lt;- df2[,2]
  KLx.dist(X, Y, k = 5)
  [1]        NaN       -Inf -0.1928958  0.0312911  0.1972085
  </code></pre>
  
  <p>The result is quite funny, none of these distances are even close to each other! My question here would be: am I applying the test correctly? if yes, why do the distances show that huge difference?</p>
  
  <p>Note: If any other tests can do the job, I am happy to try them.</p>
  
  <p>Many thanks,</p>