<h4 dir="auto">Description</h4>
<p dir="auto">I just tried running <code class="notranslate">GridSearchCV</code> on <code class="notranslate">HistGradientBoostingClassifier</code> but it crashed with a division by zero error. I was running with verbose logging. The last report from GridSearchCV indicated the following status:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[CV]  learning_rate=1, max_iter=500, max_leaf_nodes=None, tol=1e-08, total=16.8min
[CV] learning_rate=1, max_iter=500, max_leaf_nodes=31, tol=1e-06 .....
Binning 0.015 GB of data: 0.089 s
Fitting gradient boosted rounds:
[1/500] 1 tree, 31 leaves, max depth = 8, in 0.042s
[2/500] 1 tree, 31 leaves, max depth = 9, in 0.058s
...
[489/500] 1 tree, 2 leaves, max depth = 1, in 0.016s
[490/500] 
&lt;crash&gt;"><pre class="notranslate"><code class="notranslate">[CV]  learning_rate=1, max_iter=500, max_leaf_nodes=None, tol=1e-08, total=16.8min
[CV] learning_rate=1, max_iter=500, max_leaf_nodes=31, tol=1e-06 .....
Binning 0.015 GB of data: 0.089 s
Fitting gradient boosted rounds:
[1/500] 1 tree, 31 leaves, max depth = 8, in 0.042s
[2/500] 1 tree, 31 leaves, max depth = 9, in 0.058s
...
[489/500] 1 tree, 2 leaves, max depth = 1, in 0.016s
[490/500] 
&lt;crash&gt;
</code></pre></div>
<h4 dir="auto">Steps/Code to Reproduce</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from sklearn.experimental import enable_hist_gradient_boosting
from sklearn.ensemble import HistGradientBoostingClassifier

parameters={
    'learning_rate':(1,0.1,0.01),
    'max_iter':(100,500,1000),
    'max_leaf_nodes':(None,31,50),
    'tol':(1e-6,1e-7,1e-8)}
hgb = HistGradientBoostingClassifier(loss='binary_crossentropy',verbose=2)
clf=GridSearchCV(hgb, parameters, cv=5, verbose=2)
clf.fit(x,y)"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">experimental</span> <span class="pl-k">import</span> <span class="pl-s1">enable_hist_gradient_boosting</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">ensemble</span> <span class="pl-k">import</span> <span class="pl-v">HistGradientBoostingClassifier</span>

<span class="pl-s1">parameters</span><span class="pl-c1">=</span>{
    <span class="pl-s">'learning_rate'</span>:(<span class="pl-c1">1</span>,<span class="pl-c1">0.1</span>,<span class="pl-c1">0.01</span>),
    <span class="pl-s">'max_iter'</span>:(<span class="pl-c1">100</span>,<span class="pl-c1">500</span>,<span class="pl-c1">1000</span>),
    <span class="pl-s">'max_leaf_nodes'</span>:(<span class="pl-c1">None</span>,<span class="pl-c1">31</span>,<span class="pl-c1">50</span>),
    <span class="pl-s">'tol'</span>:(<span class="pl-c1">1e-6</span>,<span class="pl-c1">1e-7</span>,<span class="pl-c1">1e-8</span>)}
<span class="pl-s1">hgb</span> <span class="pl-c1">=</span> <span class="pl-v">HistGradientBoostingClassifier</span>(<span class="pl-s1">loss</span><span class="pl-c1">=</span><span class="pl-s">'binary_crossentropy'</span>,<span class="pl-s1">verbose</span><span class="pl-c1">=</span><span class="pl-c1">2</span>)
<span class="pl-s1">clf</span><span class="pl-c1">=</span><span class="pl-v">GridSearchCV</span>(<span class="pl-s1">hgb</span>, <span class="pl-s1">parameters</span>, <span class="pl-s1">cv</span><span class="pl-c1">=</span><span class="pl-c1">5</span>, <span class="pl-s1">verbose</span><span class="pl-c1">=</span><span class="pl-c1">2</span>)
<span class="pl-s1">clf</span>.<span class="pl-en">fit</span>(<span class="pl-s1">x</span>,<span class="pl-s1">y</span>)</pre></div>
<h4 dir="auto">Expected Results</h4>
<p dir="auto">Not to crash.</p>
<h4 dir="auto">Actual Results</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="ZeroDivisionError                         Traceback (most recent call last)
&lt;ipython-input-486-22ccf9e7ea94&gt; in &lt;module&gt;
      7 clf=GridSearchCV(hgb,parameters,cv=5,verbose=2)
      8 # Order:
----&gt; 9 clf.fit(x,y)
     10 clf.best_estimator_

d:\programming\python\3.7.2\lib\site-packages\sklearn\model_selection\_search.py in fit(self, X, y, groups, **fit_params)
    685                 return results
    686 
--&gt; 687             self._run_search(evaluate_candidates)
    688 
    689         # For multi-metric evaluation, store the best_index_, best_params_ and

d:\programming\python\3.7.2\lib\site-packages\sklearn\model_selection\_search.py in _run_search(self, evaluate_candidates)
   1146     def _run_search(self, evaluate_candidates):
   1147         &quot;&quot;&quot;Search all candidates in param_grid&quot;&quot;&quot;
-&gt; 1148         evaluate_candidates(ParameterGrid(self.param_grid))
   1149 
   1150 

d:\programming\python\3.7.2\lib\site-packages\sklearn\model_selection\_search.py in evaluate_candidates(candidate_params)
    664                                for parameters, (train, test)
    665                                in product(candidate_params,
--&gt; 666                                           cv.split(X, y, groups)))
    667 
    668                 if len(out) &lt; 1:

d:\programming\python\3.7.2\lib\site-packages\joblib\parallel.py in __call__(self, iterable)
    922                 self._iterating = self._original_iterator is not None
    923 
--&gt; 924             while self.dispatch_one_batch(iterator):
    925                 pass
    926 

d:\programming\python\3.7.2\lib\site-packages\joblib\parallel.py in dispatch_one_batch(self, iterator)
    757                 return False
    758             else:
--&gt; 759                 self._dispatch(tasks)
    760                 return True
    761 

d:\programming\python\3.7.2\lib\site-packages\joblib\parallel.py in _dispatch(self, batch)
    714         with self._lock:
    715             job_idx = len(self._jobs)
--&gt; 716             job = self._backend.apply_async(batch, callback=cb)
    717             # A job can complete so quickly than its callback is
    718             # called before we get here, causing self._jobs to

d:\programming\python\3.7.2\lib\site-packages\joblib\_parallel_backends.py in apply_async(self, func, callback)
    180     def apply_async(self, func, callback=None):
    181         &quot;&quot;&quot;Schedule a func to be run&quot;&quot;&quot;
--&gt; 182         result = ImmediateResult(func)
    183         if callback:
    184             callback(result)

d:\programming\python\3.7.2\lib\site-packages\joblib\_parallel_backends.py in __init__(self, batch)
    547         # Don't delay the application, to avoid keeping the input
    548         # arguments in memory
--&gt; 549         self.results = batch()
    550 
    551     def get(self):

d:\programming\python\3.7.2\lib\site-packages\joblib\parallel.py in __call__(self)
    223         with parallel_backend(self._backend, n_jobs=self._n_jobs):
    224             return [func(*args, **kwargs)
--&gt; 225                     for func, args, kwargs in self.items]
    226 
    227     def __len__(self):

d:\programming\python\3.7.2\lib\site-packages\joblib\parallel.py in &lt;listcomp&gt;(.0)
    223         with parallel_backend(self._backend, n_jobs=self._n_jobs):
    224             return [func(*args, **kwargs)
--&gt; 225                     for func, args, kwargs in self.items]
    226 
    227     def __len__(self):

d:\programming\python\3.7.2\lib\site-packages\sklearn\model_selection\_validation.py in _fit_and_score(estimator, X, y, scorer, train, test, verbose, parameters, fit_params, return_train_score, return_parameters, return_n_test_samples, return_times, return_estimator, error_score)
    512             estimator.fit(X_train, **fit_params)
    513         else:
--&gt; 514             estimator.fit(X_train, y_train, **fit_params)
    515 
    516     except Exception as e:

d:\programming\python\3.7.2\lib\site-packages\sklearn\ensemble\_hist_gradient_boosting\gradient_boosting.py in fit(self, X, y)
    255                     min_samples_leaf=self.min_samples_leaf,
    256                     l2_regularization=self.l2_regularization,
--&gt; 257                     shrinkage=self.learning_rate)
    258                 grower.grow()
    259 

d:\programming\python\3.7.2\lib\site-packages\sklearn\ensemble\_hist_gradient_boosting\grower.py in __init__(self, X_binned, gradients, hessians, max_leaf_nodes, max_depth, min_samples_leaf, min_gain_to_split, max_bins, actual_n_bins, l2_regularization, min_hessian_to_split, shrinkage)
    194         self.total_compute_hist_time = 0.  # time spent computing histograms
    195         self.total_apply_split_time = 0.  # time spent splitting nodes
--&gt; 196         self._intilialize_root(gradients, hessians, hessians_are_constant)
    197         self.n_nodes = 1
    198 

d:\programming\python\3.7.2\lib\site-packages\sklearn\ensemble\_hist_gradient_boosting\grower.py in _intilialize_root(self, gradients, hessians, hessians_are_constant)
    259             return
    260         if sum_hessians &lt; self.splitter.min_hessian_to_split:
--&gt; 261             self._finalize_leaf(self.root)
    262             return
    263 

d:\programming\python\3.7.2\lib\site-packages\sklearn\ensemble\_hist_gradient_boosting\grower.py in _finalize_leaf(self, node)
    398         &quot;&quot;&quot;
    399         node.value = -self.shrinkage * node.sum_gradients / (
--&gt; 400             node.sum_hessians + self.splitter.l2_regularization)
    401         self.finalized_leaves.append(node)
    402 

ZeroDivisionError: float division by zero"><pre class="notranslate"><span class="pl-v">ZeroDivisionError</span>                         <span class="pl-v">Traceback</span> (<span class="pl-s1">most</span> <span class="pl-s1">recent</span> <span class="pl-s1">call</span> <span class="pl-s1">last</span>)
<span class="pl-c1">&lt;</span><span class="pl-s1">ipython</span><span class="pl-c1">-</span><span class="pl-s1">input</span><span class="pl-c1">-</span><span class="pl-c1">486</span><span class="pl-c1">-</span><span class="pl-c1">22</span><span class="pl-s1">ccf9e7ea94</span><span class="pl-c1">&gt;</span> <span class="pl-s1">in</span> <span class="pl-c1">&lt;</span><span class="pl-s1">module</span><span class="pl-c1">&gt;</span>
      <span class="pl-c1">7</span> <span class="pl-s1">clf</span><span class="pl-c1">=</span><span class="pl-v">GridSearchCV</span>(<span class="pl-s1">hgb</span>,<span class="pl-s1">parameters</span>,<span class="pl-s1">cv</span><span class="pl-c1">=</span><span class="pl-c1">5</span>,<span class="pl-s1">verbose</span><span class="pl-c1">=</span><span class="pl-c1">2</span>)
      <span class="pl-c1">8</span> <span class="pl-c"># Order:</span>
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">9</span> <span class="pl-s1">clf</span>.<span class="pl-en">fit</span>(<span class="pl-s1">x</span>,<span class="pl-s1">y</span>)
     <span class="pl-c1">10</span> <span class="pl-s1">clf</span>.<span class="pl-s1">best_estimator_</span>

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\s<span class="pl-s1">klearn</span>\m<span class="pl-s1">odel_selection</span>\_<span class="pl-s1">search</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">fit</span>(<span class="pl-s1">self</span>, <span class="pl-v">X</span>, <span class="pl-s1">y</span>, <span class="pl-s1">groups</span>, <span class="pl-c1">**</span><span class="pl-s1">fit_params</span>)
    <span class="pl-c1">685</span>                 <span class="pl-k">return</span> <span class="pl-s1">results</span>
    <span class="pl-c1">686</span> 
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">687</span>             <span class="pl-s1">self</span>.<span class="pl-en">_run_search</span>(<span class="pl-s1">evaluate_candidates</span>)
    <span class="pl-c1">688</span> 
    <span class="pl-c1">689</span>         <span class="pl-c"># For multi-metric evaluation, store the best_index_, best_params_ and</span>

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\s<span class="pl-s1">klearn</span>\m<span class="pl-s1">odel_selection</span>\_<span class="pl-s1">search</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">_run_search</span>(<span class="pl-s1">self</span>, <span class="pl-s1">evaluate_candidates</span>)
   <span class="pl-c1">1146</span>     <span class="pl-k">def</span> <span class="pl-en">_run_search</span>(<span class="pl-s1">self</span>, <span class="pl-s1">evaluate_candidates</span>):
   <span class="pl-c1">1147</span>         <span class="pl-s">"""Search all candidates in param_grid"""</span>
<span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">1148</span>         <span class="pl-en">evaluate_candidates</span>(<span class="pl-v">ParameterGrid</span>(<span class="pl-s1">self</span>.<span class="pl-s1">param_grid</span>))
   <span class="pl-c1">1149</span> 
   <span class="pl-c1">1150</span> 

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\s<span class="pl-s1">klearn</span>\m<span class="pl-s1">odel_selection</span>\_<span class="pl-s1">search</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">evaluate_candidates</span>(<span class="pl-s1">candidate_params</span>)
    <span class="pl-c1">664</span>                                <span class="pl-k">for</span> <span class="pl-s1">parameters</span>, (<span class="pl-s1">train</span>, <span class="pl-s1">test</span>)
    <span class="pl-c1">665</span>                                <span class="pl-c1">in</span> <span class="pl-en">product</span>(<span class="pl-s1">candidate_params</span>,
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">666</span>                                           <span class="pl-s1">cv</span>.<span class="pl-en">split</span>(<span class="pl-v">X</span>, <span class="pl-s1">y</span>, <span class="pl-s1">groups</span>)))
    <span class="pl-c1">667</span> 
    <span class="pl-c1">668</span>                 <span class="pl-k">if</span> <span class="pl-en">len</span>(<span class="pl-s1">out</span>) <span class="pl-c1">&lt;</span> <span class="pl-c1">1</span>:

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\j<span class="pl-s1">oblib</span>\p<span class="pl-s1">arallel</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">__call__</span>(<span class="pl-s1">self</span>, <span class="pl-s1">iterable</span>)
    <span class="pl-c1">922</span>                 <span class="pl-s1">self</span>.<span class="pl-s1">_iterating</span> <span class="pl-c1">=</span> <span class="pl-s1">self</span>.<span class="pl-s1">_original_iterator</span> <span class="pl-c1">is</span> <span class="pl-c1">not</span> <span class="pl-c1">None</span>
    <span class="pl-c1">923</span> 
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">924</span>             <span class="pl-s1">while</span> <span class="pl-s1">self</span>.<span class="pl-en">dispatch_one_batch</span>(<span class="pl-s1">iterator</span>):
    <span class="pl-c1">925</span>                 <span class="pl-s1">pass</span>
    <span class="pl-c1">926</span> 

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\j<span class="pl-s1">oblib</span>\p<span class="pl-s1">arallel</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">dispatch_one_batch</span>(<span class="pl-s1">self</span>, <span class="pl-s1">iterator</span>)
    <span class="pl-c1">757</span>                 <span class="pl-k">return</span> <span class="pl-c1">False</span>
    <span class="pl-c1">758</span>             <span class="pl-s1">else</span>:
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">759</span>                 <span class="pl-s1">self</span>.<span class="pl-en">_dispatch</span>(<span class="pl-s1">tasks</span>)
    <span class="pl-c1">760</span>                 <span class="pl-k">return</span> <span class="pl-c1">True</span>
    <span class="pl-c1">761</span> 

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\j<span class="pl-s1">oblib</span>\p<span class="pl-s1">arallel</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">_dispatch</span>(<span class="pl-s1">self</span>, <span class="pl-s1">batch</span>)
    <span class="pl-c1">714</span>         <span class="pl-k">with</span> <span class="pl-s1">self</span>.<span class="pl-s1">_lock</span>:
    <span class="pl-c1">715</span>             <span class="pl-s1">job_idx</span> <span class="pl-c1">=</span> <span class="pl-en">len</span>(<span class="pl-s1">self</span>.<span class="pl-s1">_jobs</span>)
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">716</span>             <span class="pl-s1">job</span> <span class="pl-c1">=</span> <span class="pl-s1">self</span>.<span class="pl-s1">_backend</span>.<span class="pl-en">apply_async</span>(<span class="pl-s1">batch</span>, <span class="pl-s1">callback</span><span class="pl-c1">=</span><span class="pl-s1">cb</span>)
    <span class="pl-c1">717</span>             <span class="pl-c"># A job can complete so quickly than its callback is</span>
    <span class="pl-c1">718</span>             <span class="pl-c"># called before we get here, causing self._jobs to</span>

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\j<span class="pl-s1">oblib</span>\_<span class="pl-s1">parallel_backends</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">apply_async</span>(<span class="pl-s1">self</span>, <span class="pl-s1">func</span>, <span class="pl-s1">callback</span>)
    <span class="pl-c1">180</span>     <span class="pl-k">def</span> <span class="pl-en">apply_async</span>(<span class="pl-s1">self</span>, <span class="pl-s1">func</span>, <span class="pl-s1">callback</span><span class="pl-c1">=</span><span class="pl-c1">None</span>):
    <span class="pl-c1">181</span>         <span class="pl-s">"""Schedule a func to be run"""</span>
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">182</span>         <span class="pl-s1">result</span> <span class="pl-c1">=</span> <span class="pl-v">ImmediateResult</span>(<span class="pl-s1">func</span>)
    <span class="pl-c1">183</span>         <span class="pl-k">if</span> <span class="pl-s1">callback</span>:
    <span class="pl-c1">184</span>             <span class="pl-en">callback</span>(<span class="pl-s1">result</span>)

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\j<span class="pl-s1">oblib</span>\_<span class="pl-s1">parallel_backends</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">__init__</span>(<span class="pl-s1">self</span>, <span class="pl-s1">batch</span>)
    <span class="pl-c1">547</span>         <span class="pl-c"># Don't delay the application, to avoid keeping the input</span>
    <span class="pl-c1">548</span>         <span class="pl-c"># arguments in memory</span>
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">549</span>         <span class="pl-s1">self</span>.<span class="pl-s1">results</span> <span class="pl-c1">=</span> <span class="pl-en">batch</span>()
    <span class="pl-c1">550</span> 
    <span class="pl-c1">551</span>     <span class="pl-s1">def</span> <span class="pl-en">get</span>(<span class="pl-s1">self</span>):

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\j<span class="pl-s1">oblib</span>\p<span class="pl-s1">arallel</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">__call__</span>(<span class="pl-s1">self</span>)
    <span class="pl-c1">223</span>         <span class="pl-s1">with</span> <span class="pl-en">parallel_backend</span>(<span class="pl-s1">self</span>.<span class="pl-s1">_backend</span>, <span class="pl-s1">n_jobs</span><span class="pl-c1">=</span><span class="pl-s1">self</span>.<span class="pl-s1">_n_jobs</span>):
    <span class="pl-c1">224</span>             <span class="pl-s1">return</span> [<span class="pl-en">func</span>(<span class="pl-c1">*</span><span class="pl-s1">args</span>, <span class="pl-c1">**</span><span class="pl-s1">kwargs</span>)
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">225</span>                     <span class="pl-s1">for</span> <span class="pl-s1">func</span>, <span class="pl-s1">args</span>, <span class="pl-s1">kwargs</span> <span class="pl-c1">in</span> <span class="pl-s1">self</span>.<span class="pl-s1">items</span>]
    <span class="pl-c1">226</span> 
    <span class="pl-c1">227</span>     <span class="pl-s1">def</span> <span class="pl-en">__len__</span>(<span class="pl-s1">self</span>):

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\j<span class="pl-s1">oblib</span>\p<span class="pl-s1">arallel</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-c1">&lt;</span><span class="pl-s1">listcomp</span><span class="pl-c1">&gt;</span>(<span class="pl-c1">.0</span>)
    <span class="pl-c1">223</span>         <span class="pl-s1">with</span> <span class="pl-en">parallel_backend</span>(<span class="pl-s1">self</span>.<span class="pl-s1">_backend</span>, <span class="pl-s1">n_jobs</span><span class="pl-c1">=</span><span class="pl-s1">self</span>.<span class="pl-s1">_n_jobs</span>):
    <span class="pl-c1">224</span>             <span class="pl-s1">return</span> [<span class="pl-en">func</span>(<span class="pl-c1">*</span><span class="pl-s1">args</span>, <span class="pl-c1">**</span><span class="pl-s1">kwargs</span>)
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">225</span>                     <span class="pl-s1">for</span> <span class="pl-s1">func</span>, <span class="pl-s1">args</span>, <span class="pl-s1">kwargs</span> <span class="pl-c1">in</span> <span class="pl-s1">self</span>.<span class="pl-s1">items</span>]
    <span class="pl-c1">226</span> 
    <span class="pl-c1">227</span>     <span class="pl-s1">def</span> <span class="pl-en">__len__</span>(<span class="pl-s1">self</span>):

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\s<span class="pl-s1">klearn</span>\m<span class="pl-s1">odel_selection</span>\_<span class="pl-s1">validation</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">_fit_and_score</span>(<span class="pl-s1">estimator</span>, <span class="pl-v">X</span>, <span class="pl-s1">y</span>, <span class="pl-s1">scorer</span>, <span class="pl-s1">train</span>, <span class="pl-s1">test</span>, <span class="pl-s1">verbose</span>, <span class="pl-s1">parameters</span>, <span class="pl-s1">fit_params</span>, <span class="pl-s1">return_train_score</span>, <span class="pl-s1">return_parameters</span>, <span class="pl-s1">return_n_test_samples</span>, <span class="pl-s1">return_times</span>, <span class="pl-s1">return_estimator</span>, <span class="pl-s1">error_score</span>)
    <span class="pl-c1">512</span>             <span class="pl-s1">estimator</span>.<span class="pl-en">fit</span>(<span class="pl-v">X_train</span>, <span class="pl-c1">**</span><span class="pl-s1">fit_params</span>)
    <span class="pl-c1">513</span>         <span class="pl-k">else</span>:
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">514</span>             <span class="pl-s1">estimator</span>.<span class="pl-en">fit</span>(<span class="pl-v">X_train</span>, <span class="pl-s1">y_train</span>, <span class="pl-c1">**</span><span class="pl-s1">fit_params</span>)
    <span class="pl-c1">515</span> 
    <span class="pl-c1">516</span>     <span class="pl-s1">except</span> <span class="pl-v">Exception</span> <span class="pl-k">as</span> <span class="pl-s1">e</span>:

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\s<span class="pl-s1">klearn</span>\e<span class="pl-s1">nsemble</span>\_<span class="pl-s1">hist_gradient_boosting</span>\g<span class="pl-s1">radient_boosting</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">fit</span>(<span class="pl-s1">self</span>, <span class="pl-v">X</span>, <span class="pl-s1">y</span>)
    <span class="pl-c1">255</span>                     <span class="pl-s1">min_samples_leaf</span><span class="pl-c1">=</span><span class="pl-s1">self</span>.<span class="pl-s1">min_samples_leaf</span>,
    <span class="pl-c1">256</span>                     <span class="pl-s1">l2_regularization</span><span class="pl-c1">=</span><span class="pl-s1">self</span>.<span class="pl-s1">l2_regularization</span>,
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">257</span>                     <span class="pl-s1">shrinkage</span><span class="pl-c1">=</span><span class="pl-s1">self</span>.<span class="pl-s1">learning_rate</span>)
    <span class="pl-c1">258</span>                 <span class="pl-s1">grower</span>.<span class="pl-en">grow</span>()
    <span class="pl-c1">259</span> 

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\s<span class="pl-s1">klearn</span>\e<span class="pl-s1">nsemble</span>\_<span class="pl-s1">hist_gradient_boosting</span>\g<span class="pl-s1">rower</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">__init__</span>(<span class="pl-s1">self</span>, <span class="pl-v">X_binned</span>, <span class="pl-s1">gradients</span>, <span class="pl-s1">hessians</span>, <span class="pl-s1">max_leaf_nodes</span>, <span class="pl-s1">max_depth</span>, <span class="pl-s1">min_samples_leaf</span>, <span class="pl-s1">min_gain_to_split</span>, <span class="pl-s1">max_bins</span>, <span class="pl-s1">actual_n_bins</span>, <span class="pl-s1">l2_regularization</span>, <span class="pl-s1">min_hessian_to_split</span>, <span class="pl-s1">shrinkage</span>)
    <span class="pl-c1">194</span>         <span class="pl-s1">self</span>.<span class="pl-s1">total_compute_hist_time</span> <span class="pl-c1">=</span> <span class="pl-c1">0.</span>  <span class="pl-c"># time spent computing histograms</span>
    <span class="pl-c1">195</span>         <span class="pl-s1">self</span>.<span class="pl-s1">total_apply_split_time</span> <span class="pl-c1">=</span> <span class="pl-c1">0.</span>  <span class="pl-c"># time spent splitting nodes</span>
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">196</span>         <span class="pl-s1">self</span>.<span class="pl-en">_intilialize_root</span>(<span class="pl-s1">gradients</span>, <span class="pl-s1">hessians</span>, <span class="pl-s1">hessians_are_constant</span>)
    <span class="pl-c1">197</span>         <span class="pl-s1">self</span>.<span class="pl-s1">n_nodes</span> <span class="pl-c1">=</span> <span class="pl-c1">1</span>
    <span class="pl-c1">198</span> 

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\s<span class="pl-s1">klearn</span>\e<span class="pl-s1">nsemble</span>\_<span class="pl-s1">hist_gradient_boosting</span>\g<span class="pl-s1">rower</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">_intilialize_root</span>(<span class="pl-s1">self</span>, <span class="pl-s1">gradients</span>, <span class="pl-s1">hessians</span>, <span class="pl-s1">hessians_are_constant</span>)
    <span class="pl-c1">259</span>             <span class="pl-k">return</span>
    <span class="pl-c1">260</span>         <span class="pl-k">if</span> <span class="pl-s1">sum_hessians</span> <span class="pl-c1">&lt;</span> <span class="pl-s1">self</span>.<span class="pl-s1">splitter</span>.<span class="pl-s1">min_hessian_to_split</span>:
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">261</span>             <span class="pl-s1">self</span>.<span class="pl-en">_finalize_leaf</span>(<span class="pl-s1">self</span>.<span class="pl-s1">root</span>)
    <span class="pl-c1">262</span>             <span class="pl-s1">return</span>
    <span class="pl-c1">263</span> 

<span class="pl-s1">d</span>:\p<span class="pl-s1">rogramming</span>\p<span class="pl-s1">ython</span>\3<span class="pl-c1">.7</span>.<span class="pl-c1">2</span>\l<span class="pl-s1">ib</span>\s<span class="pl-s1">ite</span><span class="pl-c1">-</span><span class="pl-s1">packages</span>\s<span class="pl-s1">klearn</span>\e<span class="pl-s1">nsemble</span>\_<span class="pl-s1">hist_gradient_boosting</span>\g<span class="pl-s1">rower</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">_finalize_leaf</span>(<span class="pl-s1">self</span>, <span class="pl-s1">node</span>)
    <span class="pl-c1">398</span>         """
    <span class="pl-c1">399</span>         <span class="pl-s1">node</span>.<span class="pl-s1">value</span> <span class="pl-c1">=</span> <span class="pl-c1">-</span><span class="pl-s1">self</span>.<span class="pl-s1">shrinkage</span> <span class="pl-c1">*</span> <span class="pl-s1">node</span>.<span class="pl-s1">sum_gradients</span> <span class="pl-c1">/</span> (
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">400</span>             <span class="pl-s1">node</span>.<span class="pl-s1">sum_hessians</span> <span class="pl-c1">+</span> <span class="pl-s1">self</span>.<span class="pl-s1">splitter</span>.<span class="pl-s1">l2_regularization</span>)
    <span class="pl-c1">401</span>         <span class="pl-s1">self</span>.<span class="pl-s1">finalized_leaves</span>.<span class="pl-en">append</span>(<span class="pl-s1">node</span>)
    <span class="pl-c1">402</span> 

<span class="pl-v">ZeroDivisionError</span>: <span class="pl-s1">float</span> <span class="pl-s1">division</span> <span class="pl-s1">by</span> <span class="pl-s1">zero</span></pre></div>
<h4 dir="auto">Versions</h4>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="System:
    python: 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)]
executable: D:\Programming\Python\3.7.2\python.exe
   machine: Windows-10-10.0.17763-SP0

BLAS:
    macros:
  lib_dirs:
cblas_libs: cblas

Python deps:
       pip: 19.1.1
setuptools: 40.8.0
   sklearn: 0.21.2
     numpy: 1.16.2
     scipy: 1.2.1
    Cython: None
    pandas: 0.24.2"><pre class="notranslate"><code class="notranslate">System:
    python: 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)]
executable: D:\Programming\Python\3.7.2\python.exe
   machine: Windows-10-10.0.17763-SP0

BLAS:
    macros:
  lib_dirs:
cblas_libs: cblas

Python deps:
       pip: 19.1.1
setuptools: 40.8.0
   sklearn: 0.21.2
     numpy: 1.16.2
     scipy: 1.2.1
    Cython: None
    pandas: 0.24.2
</code></pre></div>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from sklearn.datasets import fetch_openml
from sklearn.model_selection import cross_val_score
from sklearn.experimental import enable_hist_gradient_boosting
from sklearn.ensemble import HistGradientBoostingClassifier

import numpy as np

# one hundred plants - margin
bunch = fetch_openml(data_id=1491)
X = bunch.data
y = bunch.target


res = cross_val_score(HistGradientBoostingClassifier(max_iter=100, min_samples_leaf=5), X, y)
np.mean(res)"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">datasets</span> <span class="pl-k">import</span> <span class="pl-s1">fetch_openml</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">model_selection</span> <span class="pl-k">import</span> <span class="pl-s1">cross_val_score</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">experimental</span> <span class="pl-k">import</span> <span class="pl-s1">enable_hist_gradient_boosting</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">ensemble</span> <span class="pl-k">import</span> <span class="pl-v">HistGradientBoostingClassifier</span>

<span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>

<span class="pl-c"># one hundred plants - margin</span>
<span class="pl-s1">bunch</span> <span class="pl-c1">=</span> <span class="pl-en">fetch_openml</span>(<span class="pl-s1">data_id</span><span class="pl-c1">=</span><span class="pl-c1">1491</span>)
<span class="pl-v">X</span> <span class="pl-c1">=</span> <span class="pl-s1">bunch</span>.<span class="pl-s1">data</span>
<span class="pl-s1">y</span> <span class="pl-c1">=</span> <span class="pl-s1">bunch</span>.<span class="pl-s1">target</span>


<span class="pl-s1">res</span> <span class="pl-c1">=</span> <span class="pl-en">cross_val_score</span>(<span class="pl-v">HistGradientBoostingClassifier</span>(<span class="pl-s1">max_iter</span><span class="pl-c1">=</span><span class="pl-c1">100</span>, <span class="pl-s1">min_samples_leaf</span><span class="pl-c1">=</span><span class="pl-c1">5</span>), <span class="pl-v">X</span>, <span class="pl-s1">y</span>)
<span class="pl-s1">np</span>.<span class="pl-en">mean</span>(<span class="pl-s1">res</span>)</pre></div>
<p dir="auto">NaN</p>
<p dir="auto">This dataset is a bit weird in that it has 100 classes with 16 samples each. The default parameter don't work very well but we should fail more gacefully.</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/NicolasHug/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/NicolasHug">@NicolasHug</a></p>