title: Joint entropy in python
body: <p>I have two arrays:</p>
  
  <pre><code>import numpy as np
  a = np.array(['1','2','3'])
  b = np.array(['3','4','1','5'])
  </code></pre>
  
  <p>I want to calculate joint entropy. I've found some materials to make it like:</p>
  
  <pre><code>import numpy as np
  def entropy(*X):
      return = np.sum(-p * np.log2(p) if p &gt; 0 else 0 
          for p in (np.mean(reduce(np.logical_and, (predictions == c for predictions, c in zip(X, classes))))
          for classes in itertools.product(*[set(x) for x in X])))
  </code></pre>
  
  <p>Seems to work fine with <code>len(a) = len(b)</code> but it ends with error if <code>len(a) != len(b)</code></p>
  
  <p>UPD: Arrays <code>a</code> and <code>b</code> were created from exampled main input:</p>
  
  <pre><code>b:3 p1:1 p2:6 p5:7
  b:4 p1:2 p7:2
  b:1 p3:4 p5:8
  b:5 p1:3 p4:4 
  </code></pre>
  
  <p>Where array <code>a</code> was created from p1 values. So not every line consists of every <code>pK</code> but every has <code>b</code> property. I need to calculate mutual information <code>I(b,pK)</code> for each <code>pK</code>. </p>