title: KL-Divergence of two GMMs
body: <p>I have two GMMs that I used to fit two different sets of data in the same space, and I would like to calculate the KL-divergence between them.</p>
  
  <p>Currently I am using the GMMs defined in sklearn (<a href="http://scikit-learn.org/stable/modules/generated/sklearn.mixture.GMM.html" rel="noreferrer">http://scikit-learn.org/stable/modules/generated/sklearn.mixture.GMM.html</a>) and the SciPy implementation of KL-divergence (<a href="http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.stats.entropy.html" rel="noreferrer">http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.stats.entropy.html</a>)</p>
  
  <p>How would I go about doing this?  Do I want to just create tons of random points, get their probabilities on each of the two models (call them P and Q) and then use those probabilities as my input?  Or is there some more canonical way to do this within the SciPy/SKLearn environment?</p>