<h4 dir="auto">Description</h4>
<p dir="auto">Passing any estimator to the <code class="notranslate">init</code> parameter in <code class="notranslate">GradientBoostingClassifier</code> (or <code class="notranslate">GradientBoostingRegressor</code>) causes errors when fitting. I'm including one example below as a MWE but I've fiddled with this issue for a while and get a number of different errors even if I can get past one. My guess is that this part of the code (the <code class="notranslate">init</code> estimator part) hasn't been shown any love in quite a while (since I've found similar issues that date back to 2014: <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="24795572" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/2691" data-hovercard-type="issue" data-hovercard-url="/scikit-learn/scikit-learn/issues/2691/hovercard" href="https://github.com/scikit-learn/scikit-learn/issues/2691">#2691</a>). A more complete exploration can be found here: <a href="https://github.com/stoddardg/sklearn_bug_example/blob/master/Gradient%2BBoosting%2Binit%2BMWE.ipynb">https://github.com/stoddardg/sklearn_bug_example/blob/master/Gradient%2BBoosting%2Binit%2BMWE.ipynb</a></p>
<h4 dir="auto">Steps/Code to Reproduce</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegressionCV
from sklearn.ensemble import GradientBoostingClassifier

X, y = make_classification(n_samples=1000, n_features=50, n_informative=10,random_state=0)
init_clf = LogisticRegressionCV()
clf = GradientBoostingClassifier(init=init_clf)
clf.fit(X, y)
"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">datasets</span> <span class="pl-k">import</span> <span class="pl-s1">make_classification</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">linear_model</span> <span class="pl-k">import</span> <span class="pl-v">LogisticRegressionCV</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">ensemble</span> <span class="pl-k">import</span> <span class="pl-v">GradientBoostingClassifier</span>

<span class="pl-v">X</span>, <span class="pl-s1">y</span> <span class="pl-c1">=</span> <span class="pl-en">make_classification</span>(<span class="pl-s1">n_samples</span><span class="pl-c1">=</span><span class="pl-c1">1000</span>, <span class="pl-s1">n_features</span><span class="pl-c1">=</span><span class="pl-c1">50</span>, <span class="pl-s1">n_informative</span><span class="pl-c1">=</span><span class="pl-c1">10</span>,<span class="pl-s1">random_state</span><span class="pl-c1">=</span><span class="pl-c1">0</span>)
<span class="pl-s1">init_clf</span> <span class="pl-c1">=</span> <span class="pl-v">LogisticRegressionCV</span>()
<span class="pl-s1">clf</span> <span class="pl-c1">=</span> <span class="pl-v">GradientBoostingClassifier</span>(<span class="pl-s1">init</span><span class="pl-c1">=</span><span class="pl-s1">init_clf</span>)
<span class="pl-s1">clf</span>.<span class="pl-en">fit</span>(<span class="pl-v">X</span>, <span class="pl-s1">y</span>)</pre></div>
<h4 dir="auto">Actual Results</h4>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="IndexError                                Traceback (most recent call last)
&lt;ipython-input-5-f94c05274ee0&gt; in &lt;module&gt;()
      6 init_clf = LogisticRegressionCV()
      7 clf = GradientBoostingClassifier(init=init_clf)
----&gt; 8 clf.fit(train_X, train_y)

~/anaconda3/envs/eis_env/lib/python3.7/site-packages/sklearn/ensemble/gradient_boosting.py in fit(self, X, y, sample_weight, monitor)
   1463         n_stages = self._fit_stages(X, y, y_pred, sample_weight, self._rng,
   1464                                     X_val, y_val, sample_weight_val,
-&gt; 1465                                     begin_at_stage, monitor, X_idx_sorted)
   1466 
   1467         # change shape of arrays after fit (early-stopping or additional ests)

~/anaconda3/envs/eis_env/lib/python3.7/site-packages/sklearn/ensemble/gradient_boosting.py in _fit_stages(self, X, y, y_pred, sample_weight, random_state, X_val, y_val, sample_weight_val, begin_at_stage, monitor, X_idx_sorted)
   1527             y_pred = self._fit_stage(i, X, y, y_pred, sample_weight,
   1528                                      sample_mask, random_state, X_idx_sorted,
-&gt; 1529                                      X_csc, X_csr)
   1530 
   1531             # track deviance (= loss)

~/anaconda3/envs/eis_env/lib/python3.7/site-packages/sklearn/ensemble/gradient_boosting.py in _fit_stage(self, i, X, y, y_pred, sample_weight, sample_mask, random_state, X_idx_sorted, X_csc, X_csr)
   1197             loss.update_terminal_regions(tree.tree_, X, y, residual, y_pred,
   1198                                          sample_weight, sample_mask,
-&gt; 1199                                          self.learning_rate, k=k)
   1200 
   1201             # add tree to ensemble

~/anaconda3/envs/eis_env/lib/python3.7/site-packages/sklearn/ensemble/gradient_boosting.py in update_terminal_regions(self, tree, X, y, residual, y_pred, sample_weight, sample_mask, learning_rate, k)
    396             self._update_terminal_region(tree, masked_terminal_regions,
    397                                          leaf, X, y, residual,
--&gt; 398                                          y_pred[:, k], sample_weight)
    399 
    400         # update predictions (both in-bag and out-of-bag)

IndexError: too many indices for array"><pre class="notranslate"><code class="notranslate">IndexError                                Traceback (most recent call last)
&lt;ipython-input-5-f94c05274ee0&gt; in &lt;module&gt;()
      6 init_clf = LogisticRegressionCV()
      7 clf = GradientBoostingClassifier(init=init_clf)
----&gt; 8 clf.fit(train_X, train_y)

~/anaconda3/envs/eis_env/lib/python3.7/site-packages/sklearn/ensemble/gradient_boosting.py in fit(self, X, y, sample_weight, monitor)
   1463         n_stages = self._fit_stages(X, y, y_pred, sample_weight, self._rng,
   1464                                     X_val, y_val, sample_weight_val,
-&gt; 1465                                     begin_at_stage, monitor, X_idx_sorted)
   1466 
   1467         # change shape of arrays after fit (early-stopping or additional ests)

~/anaconda3/envs/eis_env/lib/python3.7/site-packages/sklearn/ensemble/gradient_boosting.py in _fit_stages(self, X, y, y_pred, sample_weight, random_state, X_val, y_val, sample_weight_val, begin_at_stage, monitor, X_idx_sorted)
   1527             y_pred = self._fit_stage(i, X, y, y_pred, sample_weight,
   1528                                      sample_mask, random_state, X_idx_sorted,
-&gt; 1529                                      X_csc, X_csr)
   1530 
   1531             # track deviance (= loss)

~/anaconda3/envs/eis_env/lib/python3.7/site-packages/sklearn/ensemble/gradient_boosting.py in _fit_stage(self, i, X, y, y_pred, sample_weight, sample_mask, random_state, X_idx_sorted, X_csc, X_csr)
   1197             loss.update_terminal_regions(tree.tree_, X, y, residual, y_pred,
   1198                                          sample_weight, sample_mask,
-&gt; 1199                                          self.learning_rate, k=k)
   1200 
   1201             # add tree to ensemble

~/anaconda3/envs/eis_env/lib/python3.7/site-packages/sklearn/ensemble/gradient_boosting.py in update_terminal_regions(self, tree, X, y, residual, y_pred, sample_weight, sample_mask, learning_rate, k)
    396             self._update_terminal_region(tree, masked_terminal_regions,
    397                                          leaf, X, y, residual,
--&gt; 398                                          y_pred[:, k], sample_weight)
    399 
    400         # update predictions (both in-bag and out-of-bag)

IndexError: too many indices for array
</code></pre></div>
<h4 dir="auto">Versions</h4>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="System
------
    python: 3.7.0 (default, Jun 28 2018, 13:15:42)  [GCC 7.2.0]
executable: /home/gstoddard/anaconda3/envs/eis_env/bin/python
   machine: Linux-3.10.0-862.14.4.el7.x86_64-x86_64-with-redhat-7.5-Maipo

BLAS
----
    macros: SCIPY_MKL_H=None, HAVE_CBLAS=None
  lib_dirs: /home/gstoddard/anaconda3/envs/eis_env/lib
cblas_libs: mkl_rt, pthread

Python deps
-----------
       pip: 18.0
setuptools: 39.2.0
   sklearn: 0.20.0
     numpy: 1.15.0
     scipy: 1.1.0
    Cython: None
    pandas: 0.23.3"><pre class="notranslate"><code class="notranslate">System
------
    python: 3.7.0 (default, Jun 28 2018, 13:15:42)  [GCC 7.2.0]
executable: /home/gstoddard/anaconda3/envs/eis_env/bin/python
   machine: Linux-3.10.0-862.14.4.el7.x86_64-x86_64-with-redhat-7.5-Maipo

BLAS
----
    macros: SCIPY_MKL_H=None, HAVE_CBLAS=None
  lib_dirs: /home/gstoddard/anaconda3/envs/eis_env/lib
cblas_libs: mkl_rt, pthread

Python deps
-----------
       pip: 18.0
setuptools: 39.2.0
   sklearn: 0.20.0
     numpy: 1.15.0
     scipy: 1.1.0
    Cython: None
    pandas: 0.23.3
</code></pre></div>
<p dir="auto">here is a tiny script which reproduces the crash.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from sklearn.datasets import load_iris
from sklearn import ensemble
from sklearn.cross_validation import train_test_split

iris = load_iris()
X, y = iris.data, iris.target
X, y = X[y &lt; 2], y[y &lt; 2]  # make it binary

X_train, X_test, y_train, y_test = train_test_split(X, y)

# Fit GBT init with RF
rf = ensemble.RandomForestClassifier()
clf = ensemble.GradientBoostingClassifier(init=rf)

clf.fit(X_train, y_train)
acc = clf.score(X_test, y_test)
print(&quot;Accuracy: {:.4f}&quot;.format(acc))"><pre class="notranslate"><code class="notranslate">from sklearn.datasets import load_iris
from sklearn import ensemble
from sklearn.cross_validation import train_test_split

iris = load_iris()
X, y = iris.data, iris.target
X, y = X[y &lt; 2], y[y &lt; 2]  # make it binary

X_train, X_test, y_train, y_test = train_test_split(X, y)

# Fit GBT init with RF
rf = ensemble.RandomForestClassifier()
clf = ensemble.GradientBoostingClassifier(init=rf)

clf.fit(X_train, y_train)
acc = clf.score(X_test, y_test)
print("Accuracy: {:.4f}".format(acc))
</code></pre></div>
<p dir="auto">It also seems that the init param in GradientBoostingClassifier is<br>
not really tested.</p>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/pprett/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/pprett">@pprett</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/glouppe/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/glouppe">@glouppe</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ogrisel/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ogrisel">@ogrisel</a></p>