<p dir="auto">Before I start I'm not sure if this is an issue of interest, however from my perspective it is a documentation concern at least.</p>
<p dir="auto">Additionally I could be using this incorrectly. Please inform me if that's the case.</p>
<h4 dir="auto">Description</h4>
<p dir="auto">I'm working with multi-label data and I've been receiving results from an json api.</p>
<p dir="auto">I got this process working in a jupyter notebook, so I wasn't certain why it didn't appear to be working when I was calling this api with more data.</p>
<h4 dir="auto">Steps/Code to Reproduce</h4>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import numpy as np
from sklearn import metrics


targets = [[0, 0, 1, 0],
 [1, 0, 1, 0],
 [1, 0, 0, 1],
 [0, 1, 0, 1],
 [0, 0, 1, 0],
 [0, 1, 0, 1],
 [0, 0, 1, 1],
 [1, 1, 0, 0],
 [0, 1, 0, 1],
 [1, 0, 0, 0],
 [0, 0, 1, 1],
 [1, 1, 0, 0],
 [0, 1, 1, 0],
 [0, 1, 0, 1],
 [1, 0, 1, 0]]

pred_classes = [[0, 0, 1, 0],
 [1, 0, 0, 0],
 [0, 0, 0, 1],
 [0, 0, 0, 1],
 [0, 1, 0, 0],
 [1, 0, 0, 0],
 [0, 0, 0, 1],
 [0, 0, 1, 0],
 [0, 1, 0, 0],
 [0, 0, 0, 1],
 [0, 0, 1, 0],
 [0, 1, 0, 0],
 [0, 1, 0, 0],
 [0, 0, 1, 0],
 [0, 0, 1, 0]]

target_names = ['alt.atheism', 'comp.graphics', 'sci.med', 'soc.religion.christian']"><pre class="notranslate"><code class="notranslate">import numpy as np
from sklearn import metrics


targets = [[0, 0, 1, 0],
 [1, 0, 1, 0],
 [1, 0, 0, 1],
 [0, 1, 0, 1],
 [0, 0, 1, 0],
 [0, 1, 0, 1],
 [0, 0, 1, 1],
 [1, 1, 0, 0],
 [0, 1, 0, 1],
 [1, 0, 0, 0],
 [0, 0, 1, 1],
 [1, 1, 0, 0],
 [0, 1, 1, 0],
 [0, 1, 0, 1],
 [1, 0, 1, 0]]

pred_classes = [[0, 0, 1, 0],
 [1, 0, 0, 0],
 [0, 0, 0, 1],
 [0, 0, 0, 1],
 [0, 1, 0, 0],
 [1, 0, 0, 0],
 [0, 0, 0, 1],
 [0, 0, 1, 0],
 [0, 1, 0, 0],
 [0, 0, 0, 1],
 [0, 0, 1, 0],
 [0, 1, 0, 0],
 [0, 1, 0, 0],
 [0, 0, 1, 0],
 [0, 0, 1, 0]]

target_names = ['alt.atheism', 'comp.graphics', 'sci.med', 'soc.religion.christian']
</code></pre></div>
<h4 dir="auto">Expected Results</h4>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="print(metrics.classification_report(targets, pred_classes, target_names=target_names))

                        precision    recall  f1-score   support
           alt.atheism       0.50      0.17      0.25         6
         comp.graphics       0.75      0.43      0.55         7
               sci.med       0.60      0.43      0.50         7
soc.religion.christian       0.75      0.43      0.55         7
           avg / total       0.66      0.37      0.47        27
"><pre class="notranslate"><code class="notranslate">print(metrics.classification_report(targets, pred_classes, target_names=target_names))

                        precision    recall  f1-score   support
           alt.atheism       0.50      0.17      0.25         6
         comp.graphics       0.75      0.43      0.55         7
               sci.med       0.60      0.43      0.50         7
soc.religion.christian       0.75      0.43      0.55         7
           avg / total       0.66      0.37      0.47        27

</code></pre></div>
<h4 dir="auto">Actual Results</h4>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="print(metrics.classification_report(targets, pred_classes, target_names=target_names))

Traceback (most recent call last):
  File &quot;/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/IPython/core/interactiveshell.py&quot;, line 2885, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File &quot;&lt;ipython-input-34-ba0a51c823fd&gt;&quot;, line 1, in &lt;module&gt;
    print(metrics.classification_report(targets, pred_classes, target_names=target_names))
  File &quot;/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py&quot;, line 1391, in classification_report
    labels = unique_labels(y_true, y_pred)
  File &quot;/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/utils/multiclass.py&quot;, line 98, in unique_labels
    raise ValueError(&quot;Unknown label type: %s&quot; % repr(ys))
ValueError: Unknown label type: ([[0, 0, 1, 0], [1, 0, 1, 0], [1, 0, 0, 1], [0, 1, 0, 1], [0, 0, 1, 0], [0, 1, 0, 1], [0, 0, 1, 1], [1, 1, 0, 0], [0, 1, 0, 1], [1, 0, 0, 0], [0, 0, 1, 1], [1, 1, 0, 0], [0, 1, 1, 0], [0, 1, 0, 1], [1, 0, 1, 0]], [[0, 0, 1, 0], [1, 0, 0, 0], [0, 0, 0, 1], [0, 0, 0, 1], [0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 1, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 1, 0]])


print(metrics.classification_report(targets, pred_classes, labels=[0, 1, 2, 3], target_names=target_names))

Traceback (most recent call last):
  File &quot;/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/IPython/core/interactiveshell.py&quot;, line 2885, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File &quot;&lt;ipython-input-35-f56c95fbee5e&gt;&quot;, line 1, in &lt;module&gt;
    print(metrics.classification_report(targets, pred_classes, labels=[0, 1, 2, 3], target_names=target_names))
  File &quot;/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py&quot;, line 1415, in classification_report
    sample_weight=sample_weight)
  File &quot;/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py&quot;, line 1003, in precision_recall_fscore_support
    y_type, y_true, y_pred = _check_targets(y_true, y_pred)
  File &quot;/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py&quot;, line 89, in _check_targets
    raise ValueError(&quot;{0} is not supported&quot;.format(y_type))
ValueError: multiclass-multioutput is not supported"><pre class="notranslate"><code class="notranslate">print(metrics.classification_report(targets, pred_classes, target_names=target_names))

Traceback (most recent call last):
  File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2885, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "&lt;ipython-input-34-ba0a51c823fd&gt;", line 1, in &lt;module&gt;
    print(metrics.classification_report(targets, pred_classes, target_names=target_names))
  File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py", line 1391, in classification_report
    labels = unique_labels(y_true, y_pred)
  File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/utils/multiclass.py", line 98, in unique_labels
    raise ValueError("Unknown label type: %s" % repr(ys))
ValueError: Unknown label type: ([[0, 0, 1, 0], [1, 0, 1, 0], [1, 0, 0, 1], [0, 1, 0, 1], [0, 0, 1, 0], [0, 1, 0, 1], [0, 0, 1, 1], [1, 1, 0, 0], [0, 1, 0, 1], [1, 0, 0, 0], [0, 0, 1, 1], [1, 1, 0, 0], [0, 1, 1, 0], [0, 1, 0, 1], [1, 0, 1, 0]], [[0, 0, 1, 0], [1, 0, 0, 0], [0, 0, 0, 1], [0, 0, 0, 1], [0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 1, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 1, 0]])


print(metrics.classification_report(targets, pred_classes, labels=[0, 1, 2, 3], target_names=target_names))

Traceback (most recent call last):
  File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2885, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "&lt;ipython-input-35-f56c95fbee5e&gt;", line 1, in &lt;module&gt;
    print(metrics.classification_report(targets, pred_classes, labels=[0, 1, 2, 3], target_names=target_names))
  File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py", line 1415, in classification_report
    sample_weight=sample_weight)
  File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py", line 1003, in precision_recall_fscore_support
    y_type, y_true, y_pred = _check_targets(y_true, y_pred)
  File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py", line 89, in _check_targets
    raise ValueError("{0} is not supported".format(y_type))
ValueError: multiclass-multioutput is not supported
</code></pre></div>
<h3 dir="auto">Solution</h3>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="targets = np.vstack(targets)
pred_classes = np.vstack(pred_classes)

print(metrics.classification_report(targets, pred_classes, target_names=target_names))
                        precision    recall  f1-score   support
           alt.atheism       0.50      0.17      0.25         6
         comp.graphics       0.75      0.43      0.55         7
               sci.med       0.60      0.43      0.50         7
soc.religion.christian       0.75      0.43      0.55         7
           avg / total       0.66      0.37      0.47        27"><pre class="notranslate"><code class="notranslate">targets = np.vstack(targets)
pred_classes = np.vstack(pred_classes)

print(metrics.classification_report(targets, pred_classes, target_names=target_names))
                        precision    recall  f1-score   support
           alt.atheism       0.50      0.17      0.25         6
         comp.graphics       0.75      0.43      0.55         7
               sci.med       0.60      0.43      0.50         7
soc.religion.christian       0.75      0.43      0.55         7
           avg / total       0.66      0.37      0.47        27
</code></pre></div>
<h3 dir="auto">Suggestions</h3>
<p dir="auto">A possible improvement would be a warning that the datastructure could be the issue.</p>
<p dir="auto">Alternatively this is outside the scope of the library and that you expect users to be aware of the data they are working with.</p>
<h4 dir="auto">Versions</h4>
<p dir="auto">Darwin-15.6.0-x86_64-i386-64bit<br>
Python 3.5.2 (default, Oct 11 2016, 05:05:28)<br>
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)]<br>
NumPy 1.12.0b1<br>
SciPy 0.18.1<br>
Scikit-Learn 0.18</p>
<p dir="auto">Please note that this is a follow up issue to <a href="https://github.com/scikit-learn/scikit-learn/issues/4701" data-hovercard-type="issue" data-hovercard-url="/scikit-learn/scikit-learn/issues/4701/hovercard">#4701</a> and <a href="https://github.com/scikit-learn/scikit-learn/issues/4197" data-hovercard-type="issue" data-hovercard-url="/scikit-learn/scikit-learn/issues/4197/hovercard">#4197</a> not a duplicate.</p>
<p dir="auto">Calling sklearn.metrics.silhouette_samples with a large data set will cause a memory MemoryError in numpy.dot. This is not an issue with numpy.dot but with the implementation of sklearn.metrics.silhouette_samples. For a data set of length N it computes the full NxN pairwise distance matrix which can clearly take a lot of memory. Mathematically all that is needed in memory is a list of the same length as the largest cluster and the rest can be done by appropriate looping.</p>
<p dir="auto">This issue is to propose a new feature where a new parameter is added to the sklearn.metrics.silhouette_samples method. This would be a boolean to indicate if one wanted to use the faster but more memory consumptive existing method or the alternative method which is better on memory but likely slower.</p>
<p dir="auto">A working version of the the new method was posted in an answer on <a href="https://stackoverflow.com/questions/47702750/memoryerror-from-sklearn-metrics-silhouette-samples" rel="nofollow">StackOverflow</a></p>