<p dir="auto"><code class="notranslate">sklearn.metrics</code> introduces <code class="notranslate">isclose()</code> in <a href="https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/metrics/ranking.py">https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/metrics/ranking.py</a> which can leave the unaware data practitioner with hours of debugging.</p>
<p dir="auto">In very unbalanced classification, probabilities/scores can be very small and yet meaningful. This however will cause unexpected missing precision_recall points due to <code class="notranslate">isclose</code> treating values within 10e-6 as equal.</p>
<p dir="auto">I'd suggest to place a warning about <code class="notranslate">isclose</code> in the documentation and also replace the absolute epsilon by a relative closeness comparison in order to avoid the problems with small probabilities in unbalanced classification.</p>
<p dir="auto">pred=[1e-10, 0, 0]<br>
sol=[1, 0, 0]<br>
metrics.roc_auc_score(sol, pred) # 0.5, wrong, 1 is correct</p>
<p dir="auto">pred=[1, 0, 0]<br>
sol=[1, 0, 0]<br>
metrics.roc_auc_score(sol, pred) # 1 correct</p>