<p dir="auto">Right now, the content of <code class="notranslate">classes_</code> and the shape of the output of <code class="notranslate">predict_proba</code> and <code class="notranslate">decision_function</code> are inconsistent between estimators.<br>
If an <a href="http://scikit-learn.org/dev/modules/multiclass.html#multiclass-and-multilabel-algorithms" rel="nofollow">estimator supports</a> multioutput multiclass, <code class="notranslate">classes_</code> will be <code class="notranslate">[[0, 1]] * n_outputs</code> and <code class="notranslate">predict_proba</code> will return an object array of length <code class="notranslate">n_outputs</code> where each is <code class="notranslate">(n_samples, 2)</code>.<br>
If the estimator supports multi-label classification, but not multioutput multiclass, <code class="notranslate">classes_</code> will be <code class="notranslate">np.arange(n_outputs)</code> and <code class="notranslate">predict_proba</code> will be <code class="notranslate">(n_samples, n_outputs)</code>.</p>
<p dir="auto">That leads to issues such as <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="223399887" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/8773" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/8773/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/8773">#8773</a>, but also runs counter to the sklearn promise of a consistent API.<br>
Given that even basic API like the <code class="notranslate">score</code> method is broken on multioutput multiclass (see <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="244101623" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/9414" data-hovercard-type="issue" data-hovercard-url="/scikit-learn/scikit-learn/issues/9414/hovercard" href="https://github.com/scikit-learn/scikit-learn/issues/9414">#9414</a>), I'm not sure it's worth keeping this maintenance burden.</p>
<p dir="auto">Alternatively we could detect in the multioutput multiclass classifiers that a problem is actually multilabel and use the appropriate shapes for that.</p>
<p dir="auto">Either way, it's an incompatible change that's tricky to do with a deprecation cycle :-/</p>
<p dir="auto">In <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="855155631" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/19859" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/19859/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/19859">#19859</a>, I am heading toward addressing some issue already reported long time ago there: <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="19617817" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/2451" data-hovercard-type="issue" data-hovercard-url="/scikit-learn/scikit-learn/issues/2451/hovercard" href="https://github.com/scikit-learn/scikit-learn/issues/2451">#2451</a></p>
<p dir="auto">We have a range of classifier supporting multilabel classification. To give a bit of feedback, I recall what a multilabel indicator matrix look like:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="In [1]: from sklearn.datasets import make_multilabel_classification

In [2]: X, y = make_multilabel_classification(n_samples=10)

In [3]: y
Out[3]: 
array([[0, 0, 0, 0, 0],
       [0, 0, 0, 0, 1],
       [0, 1, 0, 0, 1],
       [0, 1, 0, 0, 1],
       [1, 1, 0, 0, 0],
       [0, 1, 1, 0, 0],
       [0, 1, 0, 0, 0],
       [0, 0, 1, 0, 0],
       [0, 1, 0, 0, 0],
       [0, 1, 1, 1, 1]])"><pre class="notranslate"><span class="pl-v">In</span> [<span class="pl-c1">1</span>]: <span class="pl-s1">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">datasets</span> <span class="pl-s1">import</span> <span class="pl-s1">make_multilabel_classification</span>

<span class="pl-v">In</span> [<span class="pl-c1">2</span>]: <span class="pl-v">X</span>, <span class="pl-s1">y</span> <span class="pl-c1">=</span> <span class="pl-en">make_multilabel_classification</span>(<span class="pl-s1">n_samples</span><span class="pl-c1">=</span><span class="pl-c1">10</span>)

<span class="pl-v">In</span> [<span class="pl-c1">3</span>]: <span class="pl-s1">y</span>
<span class="pl-v">Out</span>[<span class="pl-c1">3</span>]: 
<span class="pl-en">array</span>([[<span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>],
       [<span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">1</span>],
       [<span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">1</span>],
       [<span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">1</span>],
       [<span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>],
       [<span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>],
       [<span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>],
       [<span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>],
       [<span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>],
       [<span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>]])</pre></div>
<p dir="auto">Each column contain 0/1 entries and you can have several 1 for each row. This type of classification is a special case of <code class="notranslate">multiclass-multioutput</code> where each column will not be limited to 0/1 entries but to multiclass labels.</p>
<p dir="auto">We have some issue with consistency in scikit-learn when calling the method <code class="notranslate">predict_proba</code> with classifiers handling these type of data.</p>
<h2 dir="auto">Current situation in scikit-learn</h2>
<p dir="auto">We have a couple of classifiers that handle multilabel targets: tree-based models, neighbors, MLP, and ridge. In addition, the tree-based models and neighbors classifiers handle <code class="notranslate">multiclass-multioutput</code>.</p>
<p dir="auto">In a <code class="notranslate">multiclass-multioutput</code> problem, <code class="notranslate">classifier.predict_proba</code> return a list of n_outputs containing arrays of shape (n_samples, n_classes). Since multilabel classification is a specific case of <code class="notranslate">multiclass-multioutput</code>, <code class="notranslate">classifier.predict_proba</code> will return a list of n_outputs containing arrays of shape (n_samples, 2). This should be an historical choice.</p>
<p dir="auto">MLP and ridge (currently broken - see <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="856383991" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/19869" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/19869/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/19869">#19869</a>) do not support <code class="notranslate">multiclass-multioutput</code>. There internal design make that it is easy to output an array of shape (n_samples, n_outputs) containing the probability of the positive class only when calling <code class="notranslate">classifier.predict_proba</code>.</p>
<p dir="auto">Therefore we have two prediction format for the classifier. This RFC intends at taking a decision regarding this discrepancy.</p>
<h2 dir="auto">Potential solutions</h2>
<h3 dir="auto">1. Keep the outputs as is</h3>
<p dir="auto">We can decide to keep the output as we currently do. One can argue that by supporting <code class="notranslate">multiclass-multioutput</code> it is natural to keep the extensive format list of arrays while being efficient with arrays only for classifiers supporting <code class="notranslate">multilabel-indicator</code> targets.<br>
It would require to make the documentation to be explicit.</p>
<h3 dir="auto">2. Make output consistent</h3>
<h4 dir="auto">2.1 By adding an estimator parameter</h4>
<p dir="auto">We could add an estimator parameter allowing to switch from one representation to another.</p>
<h3 dir="auto">2.2 By adding a parameter to the <code class="notranslate">predict_proba</code> method</h3>
<p dir="auto">We could add a parameter to the <code class="notranslate">predict_proba</code> method to switch the output of the representation. It might still imply a classifier to store which type of target where fitted to distinguish between <code class="notranslate">multiclass-multioutput</code> and <code class="notranslate">multilabel-indicator</code> at predict.</p>
<h3 dir="auto">2.3 By forcing all estimators to a specific format</h3>
<p dir="auto">In addition, we could enforce a specific format such that by default all classifiers return the same output format. In this case, it requires some deprecation cycles.</p>