title: How to calculate entropy of joint distribution?
body: <p>There are two random variables <code>X</code> and <code>Y</code> and their joint probability <code>Posterior_P</code>(discrete, 2D-array), I got the entropy of posterior probability by using <a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.entropy.html" rel="nofollow noreferrer">scipy.stats.entropy</a>, which returns an array with dimension <code>len(X)</code> because the default axis = 0. I am totally new to entropy so please correct me if I confused certain definitions. 
  I created a fairly simple example to testify:</p>
  
  <pre><code>x = np.arange(-2,2,0.01)
  y = np.arange(-2,2,0.01)
  
  X,Y = np.meshgrid(x,y)
  
  # support area
  sup = np.dstack((X,Y))
  
  distribution = multivariate_normal(mean=[0,0],cov=[[1,0],[0,1]])
  
  # This is what I want, returns a value = 2.83
  entropy_0 = distribution.entropy()
  
  # calculate from 2-D array PDF 
  # returns an array [8.4605,8.4605,...,8.4605] with a size of len(X)
  pdf = distribution.pdf(sup)
  entropy_1 = entropy(pdf,base=2)
  </code></pre>
  
  <p>What's the difference between them and how can I get the overall entropy from joint density which supposed to be a value using <a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.entropy.html" rel="nofollow noreferrer">scipy.stats.entropy</a>?</p>