<p dir="auto">GradientBoostingClassifier does not work correctly, when the BasePredictor (init) is set to something like linear_model.LogisticRegression().</p>
<p dir="auto">I thinks the source of the error is in in BaseGradientBoosting.fit() where<br>
# init predictions<br>
y_pred = self.init_.predict(X)</p>
<p dir="auto">y_pred should be initialized by class probabilities and not with the winner class.</p>
<p dir="auto">Replacing y_pred = self.init_.predict(X) with y_pred = self.init_.predict_proba(X) can solve the problem.</p>
<p dir="auto">here is a tiny script which reproduces the crash.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from sklearn.datasets import load_iris
from sklearn import ensemble
from sklearn.cross_validation import train_test_split

iris = load_iris()
X, y = iris.data, iris.target
X, y = X[y &lt; 2], y[y &lt; 2]  # make it binary

X_train, X_test, y_train, y_test = train_test_split(X, y)

# Fit GBT init with RF
rf = ensemble.RandomForestClassifier()
clf = ensemble.GradientBoostingClassifier(init=rf)

clf.fit(X_train, y_train)
acc = clf.score(X_test, y_test)
print(&quot;Accuracy: {:.4f}&quot;.format(acc))"><pre class="notranslate"><code class="notranslate">from sklearn.datasets import load_iris
from sklearn import ensemble
from sklearn.cross_validation import train_test_split

iris = load_iris()
X, y = iris.data, iris.target
X, y = X[y &lt; 2], y[y &lt; 2]  # make it binary

X_train, X_test, y_train, y_test = train_test_split(X, y)

# Fit GBT init with RF
rf = ensemble.RandomForestClassifier()
clf = ensemble.GradientBoostingClassifier(init=rf)

clf.fit(X_train, y_train)
acc = clf.score(X_test, y_test)
print("Accuracy: {:.4f}".format(acc))
</code></pre></div>
<p dir="auto">It also seems that the init param in GradientBoostingClassifier is<br>
not really tested.</p>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/pprett/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/pprett">@pprett</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/glouppe/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/glouppe">@glouppe</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ogrisel/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ogrisel">@ogrisel</a></p>