<h4 dir="auto">Description</h4>
<p dir="auto">I think scoring of cross_val_score (also GridSearchCV and so on) is not good especially with cv={# of samples} (i.e. LeaveOneOut). In evaluating R2 or MAE values, mean of y is calculated with respect to each CV model in cross_val_score, but calculation of mean should be performed on all y. The difference become conspicuous when we use Leave One Out. Then mean of y with respect to each model equals to just y since number of test sample is one. Consequently, R2 value become zero.<br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/50cf4302360705646ffa6fc5aced72c98e32de48c1d0fde7447b246e750297e5/68747470733a2f2f6c617465782e636f6465636f67732e636f6d2f6769662e6c617465783f525e325f7b43567d3d312d5c667261637b5c73756d5f7b697d5e7b4e7d28795f692d5c6861747b797d5f69295e327d7b5c73756d5f7b697d5e7b4e7d28795f692d5c6261727b797d295e327d"><img src="https://camo.githubusercontent.com/50cf4302360705646ffa6fc5aced72c98e32de48c1d0fde7447b246e750297e5/68747470733a2f2f6c617465782e636f6465636f67732e636f6d2f6769662e6c617465783f525e325f7b43567d3d312d5c667261637b5c73756d5f7b697d5e7b4e7d28795f692d5c6861747b797d5f69295e327d7b5c73756d5f7b697d5e7b4e7d28795f692d5c6261727b797d295e327d" data-canonical-src="https://latex.codecogs.com/gif.latex?R^2_{CV}=1-\frac{\sum_{i}^{N}(y_i-\hat{y}_i)^2}{\sum_{i}^{N}(y_i-\bar{y})^2}" style="max-width: 100%;"></a></p>
<h4 dir="auto">Example code to Reproduce</h4>
<p dir="auto">PLS regression with Boston data set</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from sklearn import datasets,cross_decomposition
from sklearn.model_selection import cross_val_score,cross_val_predict
from sklearn.metrics import r2_score

# Boston data set
boston = datasets.load_boston()
X,y = boston.data,boston.target

pls = cross_decomposition.PLSRegression(10)
r2cv_1 = cross_val_score(pls,X,y,scoring='r2',cv=X.shape[0]).mean()
r2cv_2 = r2_score(y,cross_val_predict(pls,X,y,cv=X.shape[0]))
print('r2cv(cross_val_score   + mean)    : {0:8.4f}\nr2cv(cross_val_predict + r2_score):{1:8.4f}'.format(r2cv_1,r2cv_2))"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">sklearn</span> <span class="pl-k">import</span> <span class="pl-s1">datasets</span>,<span class="pl-s1">cross_decomposition</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">model_selection</span> <span class="pl-k">import</span> <span class="pl-s1">cross_val_score</span>,<span class="pl-s1">cross_val_predict</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">metrics</span> <span class="pl-k">import</span> <span class="pl-s1">r2_score</span>

<span class="pl-c"># Boston data set</span>
<span class="pl-s1">boston</span> <span class="pl-c1">=</span> <span class="pl-s1">datasets</span>.<span class="pl-en">load_boston</span>()
<span class="pl-v">X</span>,<span class="pl-s1">y</span> <span class="pl-c1">=</span> <span class="pl-s1">boston</span>.<span class="pl-s1">data</span>,<span class="pl-s1">boston</span>.<span class="pl-s1">target</span>

<span class="pl-s1">pls</span> <span class="pl-c1">=</span> <span class="pl-s1">cross_decomposition</span>.<span class="pl-v">PLSRegression</span>(<span class="pl-c1">10</span>)
<span class="pl-s1">r2cv_1</span> <span class="pl-c1">=</span> <span class="pl-en">cross_val_score</span>(<span class="pl-s1">pls</span>,<span class="pl-v">X</span>,<span class="pl-s1">y</span>,<span class="pl-s1">scoring</span><span class="pl-c1">=</span><span class="pl-s">'r2'</span>,<span class="pl-s1">cv</span><span class="pl-c1">=</span><span class="pl-v">X</span>.<span class="pl-s1">shape</span>[<span class="pl-c1">0</span>]).<span class="pl-en">mean</span>()
<span class="pl-s1">r2cv_2</span> <span class="pl-c1">=</span> <span class="pl-en">r2_score</span>(<span class="pl-s1">y</span>,<span class="pl-en">cross_val_predict</span>(<span class="pl-s1">pls</span>,<span class="pl-v">X</span>,<span class="pl-s1">y</span>,<span class="pl-s1">cv</span><span class="pl-c1">=</span><span class="pl-v">X</span>.<span class="pl-s1">shape</span>[<span class="pl-c1">0</span>]))
<span class="pl-en">print</span>(<span class="pl-s">'r2cv(cross_val_score   + mean)    : {0:8.4f}<span class="pl-cce">\n</span>r2cv(cross_val_predict + r2_score):{1:8.4f}'</span>.<span class="pl-en">format</span>(<span class="pl-s1">r2cv_1</span>,<span class="pl-s1">r2cv_2</span>))</pre></div>
<h4 dir="auto">Results</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="r2cv(cross_val_score   + mean)    :   0.0000
r2cv(cross_val_predict + r2_score):   0.7190"><pre class="notranslate"><span class="pl-en">r2cv</span>(<span class="pl-s1">cross_val_score</span>   <span class="pl-c1">+</span> <span class="pl-s1">mean</span>)    :   <span class="pl-c1">0.0000</span>
<span class="pl-en">r2cv</span>(<span class="pl-s1">cross_val_predict</span> <span class="pl-c1">+</span> <span class="pl-s1">r2_score</span>):   <span class="pl-c1">0.7190</span></pre></div>
<p dir="auto">R2cv value is expected to be 0.7190 but returned value is zero with cross_val_score.</p>
<h4 dir="auto">Versions</h4>
<p dir="auto">Python 3.5.2 (default, Nov 23 2017, 16:37:01)<br>
[GCC 5.4.0 20160609]<br>
NumPy 1.14.5<br>
SciPy 1.1.0<br>
Scikit-Learn 0.19.1</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from sklearn.datasets import make_regression
from sklearn.cross_validation import cross_val_score, LeaveOneOut
from sklearn.linear_model import Ridge

X, y, coef_ = make_regression(random_state=42, noise=1, n_samples=200, coef=True)
cross_val_score(Ridge(), X, y, cv=LeaveOneOut(len(X)))"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">datasets</span> <span class="pl-k">import</span> <span class="pl-s1">make_regression</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">cross_validation</span> <span class="pl-k">import</span> <span class="pl-s1">cross_val_score</span>, <span class="pl-v">LeaveOneOut</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">linear_model</span> <span class="pl-k">import</span> <span class="pl-v">Ridge</span>

<span class="pl-v">X</span>, <span class="pl-s1">y</span>, <span class="pl-s1">coef_</span> <span class="pl-c1">=</span> <span class="pl-en">make_regression</span>(<span class="pl-s1">random_state</span><span class="pl-c1">=</span><span class="pl-c1">42</span>, <span class="pl-s1">noise</span><span class="pl-c1">=</span><span class="pl-c1">1</span>, <span class="pl-s1">n_samples</span><span class="pl-c1">=</span><span class="pl-c1">200</span>, <span class="pl-s1">coef</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)
<span class="pl-en">cross_val_score</span>(<span class="pl-v">Ridge</span>(), <span class="pl-v">X</span>, <span class="pl-s1">y</span>, <span class="pl-s1">cv</span><span class="pl-c1">=</span><span class="pl-v">LeaveOneOut</span>(<span class="pl-en">len</span>(<span class="pl-v">X</span>)))</pre></div>
<blockquote>
<p dir="auto">array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,<br>
0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,<br>
0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,<br>
0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,<br>
0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.])</p>
</blockquote>
<p dir="auto">Maybe the single sample in the LOO is not interpreted correctly? :-/</p>