<p dir="auto"><a href="http://scikit-learn.org/dev/modules/clustering.html#mutual-information-based-scores" rel="nofollow">http://scikit-learn.org/dev/modules/clustering.html#mutual-information-based-scores</a></p>
<p dir="auto">All are between zero and one, but there is no concise explanation of the difference between the three, except that NMI is used in the literature and AMI is used more recently.</p>
<p dir="auto">There is a nice explanation in the IR book:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$I( \Omega ; \mathbb{C} )$ in Equation 184 measures the amount of information by which our knowledge about the classes increases when we are told what the clusters are. The minimum of $I( \Omega ; \mathbb{C} )$ is 0 if the clustering is random with respect to class membership. In that case, knowing that a document is in a particular cluster does not give us any new information about what its class might be. Maximum mutual information is reached for a clustering $\Omega_{exact}$ that perfectly recreates the classes - but also if clusters in $\Omega_{exact}$ are further subdivided into smaller clusters (Exercise 16.7 ). In particular, a clustering with $K=N$ one-document clusters has maximum MI. So MI has the same problem as purity: it does not penalize large cardinalities and thus does not formalize our bias that, other things being equal, fewer clusters are better.

The normalization by the denominator $[H(\Omega )+H(\mathbb{C} )]/2$ in Equation 183 fixes this problem since entropy tends to increase with the number of clusters. For example, $H(\Omega)$ reaches its maximum $\log N$ for $K=N$, which

ensures that NMI is low for $K=N$. Because NMI is normalized, we can use it to compare clusterings with different numbers of clusters. The particular form of the denominator is chosen because $[H(\Omega )+H(\mathbb{C} )]/2$ is a tight upper bound on $I( \Omega ; \mathbb{C} )$ (Exercise 16.7 ). Thus, NMI is always a number between 0 and 1. "><pre class="notranslate"><code class="notranslate">$I( \Omega ; \mathbb{C} )$ in Equation 184 measures the amount of information by which our knowledge about the classes increases when we are told what the clusters are. The minimum of $I( \Omega ; \mathbb{C} )$ is 0 if the clustering is random with respect to class membership. In that case, knowing that a document is in a particular cluster does not give us any new information about what its class might be. Maximum mutual information is reached for a clustering $\Omega_{exact}$ that perfectly recreates the classes - but also if clusters in $\Omega_{exact}$ are further subdivided into smaller clusters (Exercise 16.7 ). In particular, a clustering with $K=N$ one-document clusters has maximum MI. So MI has the same problem as purity: it does not penalize large cardinalities and thus does not formalize our bias that, other things being equal, fewer clusters are better.

The normalization by the denominator $[H(\Omega )+H(\mathbb{C} )]/2$ in Equation 183 fixes this problem since entropy tends to increase with the number of clusters. For example, $H(\Omega)$ reaches its maximum $\log N$ for $K=N$, which

ensures that NMI is low for $K=N$. Because NMI is normalized, we can use it to compare clusterings with different numbers of clusters. The particular form of the denominator is chosen because $[H(\Omega )+H(\mathbb{C} )]/2$ is a tight upper bound on $I( \Omega ; \mathbb{C} )$ (Exercise 16.7 ). Thus, NMI is always a number between 0 and 1. 
</code></pre></div>
<p dir="auto">Basically (if I understand correctly): MI is perfect if one is a sub-partition of the other. NMI "fixes" that and penalizes overpartioning by putting entropy in the denominator. But NMI still depends on the number of clusters and samples.</p>
<p dir="auto">There's this statement here in the user guide:</p>
<blockquote>
<p dir="auto">Random (uniform) label assignments have a AMI score close to 0.0 for any value of n_clusters and n_samples (which is not the case for raw Mutual Information or the V-measure for instance).</p>
</blockquote>
<p dir="auto">I think this should rather say NMI instead or raw MI (which I think would still be correct but better explain the point).<br>
Maybe adding some examples either inline or create a separate example would be interesting, where change in n_samples and n_clusters is demonstrated for the different cluster measures for partitions that should intuitively be similar, i.e. by just duplicating each point.</p>
<p dir="auto">We wrote the release highlights as a gallery example in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="487163914" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/14846" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/14846/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/14846">#14846</a> (for lack of alternative propositions)</p>
<p dir="auto">I just discovered this nice post on the <a href="https://scikit-learn.fondation-inria.fr/time-to-come-out/" rel="nofollow">scikit-learn foundation</a> website, which is more detailed but largely overlapping.</p>
<p dir="auto">So I'm raising the question again: where do we put the release highlights?</p>
<p dir="auto">I think it's best to have one, rather than multiple documents of the sort:</p>
<ul dir="auto">
<li>It's less confusing for users to have a single source of info</li>
<li>It's better for us not to duplicate work</li>
<li>IMHO, the post could have benefited from some peer review. There are a few approximations e.g. in the description of the permutation importance.</li>
</ul>
<p dir="auto">I'm happy to go with the foundation website from now on, but it feels strange to have multiple versions of the same thing</p>