title: How do I find the KL Divergence of samples from two 2D distributions?
body: <p>Suppose I had two 2D sets of 1000 samples that look something like this:</p>
  
  <p><a href="https://i.stack.imgur.com/hf4l4.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/hf4l4.png" alt="enter image description here"></a></p>
  
  <p>I'd like to have a metric for the amount of difference between the distributions and thought the KL divergence would be suitable. </p>
  
  <p>I've been looking at sp.stats.entropy(), however from this answer: </p>
  
  <p><a href="https://stackoverflow.com/questions/26743201/interpreting-scipy-stats-entropy-values">Interpreting scipy.stats.entropy values</a>
  it appears I need to convert it to a pdf first. How can one do this using a 4 1D arrays?</p>
  
  <p>The example data above was generated as follows: </p>
  
  <pre><code>dist1_x = np.random.normal(0, 10, 1000)
  dist1_y = np.random.normal(0, 5, 1000)
  
  dist2_x = np.random.normal(3, 10, 1000)
  dist2_y = np.random.normal(4, 5, 1000)
  
  plt.scatter(dist1_x, dist1_y)
  plt.scatter(dist2_x, dist2_y)
  plt.show()
  </code></pre>
  
  <p>For my real data I only have the samples, not the distribution from which they came (although if need be one could calculate the mean and variance and assume Gaussian).
  Is it possible to calculate the KL divergence like this?</p>