<h3 dir="auto">Reason:</h3>
<p dir="auto">It allows the user to compute these metrics on soft classifications. E.g. let Y be a (n,k) ndarray, where each row is a probability distribution over classes. Let C be a (n,c) ground truth ndarray, each row is a probability distribution over classes. Then A = np.dot(Y.T, C) is the expected contingency matrix.</p>
<h3 dir="auto">Example of a function where it's currently implemented:</h3>
<p dir="auto"><code class="notranslate">mutual_info_score</code></p>
<h3 dir="auto">Functions for which it should be implemented:</h3>
<p dir="auto"><code class="notranslate">adjusted_rand_score</code><br>
<code class="notranslate">fowlkes_mallows_score </code><br>
<code class="notranslate">normalized_mutual_info_score</code><br>
<code class="notranslate">adjusted_mutual_info_score</code><br>
<code class="notranslate">homogeneity_completeness_v_measure</code></p>
<h3 dir="auto">Discussion</h3>
<p dir="auto">Is there an architectural or semantic reason for why we wouldn't want to allow custom contingency matrices? Otherwise this seems like an easy update.</p>
<p dir="auto">Supervised clustering metrics (at least where there is a fixed set of samples, unlike in coreference resolution with system mentions) can usually be calculated from the contingency matrix alone. I think passing <code class="notranslate">y_true=contingency, y_pred=None</code> should trigger this behaviour, saving computational time when calculating multiple measures.</p>