<h4 dir="auto">Description</h4>

<p dir="auto">Unfortunately, I am not 100% sure, that this is a bug, but the LeaveOneOut cross-validation is at least not behaving as I'd expect.</p>
<h4 dir="auto">Steps/Code to Reproduce</h4>

<p dir="auto">I use this matrix:<br>
R  A  B  C  D  E  F  G  H<br>
0   149  1  0  0  0  0  0  1  0<br>
1    98  0  1  0  0  0  0  1  0<br>
2    72  0  0  1  0  0  0  1  0<br>
3    74  0  0  0  1  0  0  1  0<br>
4   124  1  0  0  0  0  0  0  1<br>
5    71  0  1  0  0  0  0  0  1<br>
6    53  0  0  1  0  0  0  0  1<br>
7    64  0  0  0  1  0  0  0  1<br>
8   186  1  0  0  0  1  1  1  0<br>
9   127  0  1  0  0  1  1  1  0<br>
10  121  0  0  1  0  1  1  1  0<br>
11  104  0  0  0  1  1  1  1  0<br>
12   98  1  0  0  0  0  1  1  1<br>
13   64  0  1  0  0  0  1  1  1<br>
14   38  0  0  1  0  0  1  1  1<br>
15   17  0  0  0  1  0  1  1  1</p>
<p dir="auto">and this code to fit R by using A-H:</p>
<p dir="auto">import pandas as pd<br>
import numpy as np<br>
import sklearn<br>
from sklearn.cross_decomposition import PLSRegression<br>
from sklearn.metrics import mean_squared_error<br>
from math import sqrt<br>
from sklearn.model_selection import LeaveOneOut<br>
from sklearn.model_selection import cross_val_score</p>
<p dir="auto">df = pd.read_csv('PSLR.csv', delimiter=';') # I read the matrix from file. This is the matrix given above<br>
y = df['R']<br>
X = df[['A','B','C','D','E','F','G','H']]<br>
pls2 = PLSRegression(n_components=3)</p>
<p dir="auto">num_folds = 3<br>
num_instances = len(X)</p>
<p dir="auto">loocv = LeaveOneOut()<br>
results = cross_val_score(pls2, X, y, cv=loocv)<br>
print("Accuracy: %.3f%% (%.3f%%)" % (results.mean()*100.0, results.std()*100.0))<br>
print(results)</p>
<h4 dir="auto">Expected Results</h4>

<p dir="auto">I expected to get a numerical result from the cross validation</p>
<h4 dir="auto">Actual Results</h4>

<p dir="auto">Accuracy: 0.000% (0.000%)<br>
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]</p>
<h4 dir="auto">Versions</h4>

<p dir="auto">System:<br>
python: 3.7.0 (default, Oct  2 2018, 09:18:58)  [Clang 10.0.0 (clang-1000.11.45.2)]<br>
executable: /usr/local/opt/python/bin/python3.7<br>
machine: Darwin-18.2.0-x86_64-i386-64bit</p>
<p dir="auto">BLAS:<br>
macros: NO_ATLAS_INFO=3, HAVE_CBLAS=None<br>
lib_dirs:<br>
cblas_libs: cblas</p>
<p dir="auto">Python deps:<br>
pip: 18.1<br>
setuptools: 40.4.3<br>
sklearn: 0.20.1<br>
numpy: 1.15.4<br>
scipy: 1.1.0<br>
Cython: None<br>
pandas: 0.23.4</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from sklearn.datasets import make_regression
from sklearn.cross_validation import cross_val_score, LeaveOneOut
from sklearn.linear_model import Ridge

X, y, coef_ = make_regression(random_state=42, noise=1, n_samples=200, coef=True)
cross_val_score(Ridge(), X, y, cv=LeaveOneOut(len(X)))"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">datasets</span> <span class="pl-k">import</span> <span class="pl-s1">make_regression</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">cross_validation</span> <span class="pl-k">import</span> <span class="pl-s1">cross_val_score</span>, <span class="pl-v">LeaveOneOut</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">linear_model</span> <span class="pl-k">import</span> <span class="pl-v">Ridge</span>

<span class="pl-v">X</span>, <span class="pl-s1">y</span>, <span class="pl-s1">coef_</span> <span class="pl-c1">=</span> <span class="pl-en">make_regression</span>(<span class="pl-s1">random_state</span><span class="pl-c1">=</span><span class="pl-c1">42</span>, <span class="pl-s1">noise</span><span class="pl-c1">=</span><span class="pl-c1">1</span>, <span class="pl-s1">n_samples</span><span class="pl-c1">=</span><span class="pl-c1">200</span>, <span class="pl-s1">coef</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)
<span class="pl-en">cross_val_score</span>(<span class="pl-v">Ridge</span>(), <span class="pl-v">X</span>, <span class="pl-s1">y</span>, <span class="pl-s1">cv</span><span class="pl-c1">=</span><span class="pl-v">LeaveOneOut</span>(<span class="pl-en">len</span>(<span class="pl-v">X</span>)))</pre></div>
<blockquote>
<p dir="auto">array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,<br>
0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,<br>
0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,<br>
0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,<br>
0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.])</p>
</blockquote>
<p dir="auto">Maybe the single sample in the LOO is not interpreted correctly? :-/</p>