<h4 dir="auto">Description</h4>

<p dir="auto">I'm running GridSearch for Gradient Boosting algorithm and for some parameters it just hangs after 10-15 minutes on my laptop. First iterations produce the python processes according to CPU number (8 in my case) with the visible workload and then the processes are gone, no progress reported. I've found the similar issue <a href="https://github.com/scikit-learn/scikit-learn/issues/3605" data-hovercard-type="issue" data-hovercard-url="/scikit-learn/scikit-learn/issues/3605/hovercard">GridSearchCV processes hang with n_jobs</a> however it's closed so I've decided to open new issue.</p>
<h4 dir="auto">Steps/Code to Reproduce</h4>
<p dir="auto">My data setup:<br>
It's <a href="https://archive.ics.uci.edu/ml/datasets/Census+Income" rel="nofollow">Census data</a> from UCI repository<br>
Originally there are 13 features:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="RangeIndex: 45222 entries, 0 to 45221
Data columns (total 14 columns):
age                45222 non-null int64
workclass          45222 non-null object
education_level    45222 non-null object
education-num      45222 non-null float64
marital-status     45222 non-null object
occupation         45222 non-null object
relationship       45222 non-null object
race               45222 non-null object
sex                45222 non-null object
capital-gain       45222 non-null float64
capital-loss       45222 non-null float64
hours-per-week     45222 non-null float64
native-country     45222 non-null object
income             45222 non-null object
dtypes: float64(4), int64(1), object(9)
memory usage: 4.8+ MB"><pre class="notranslate"><code class="notranslate">RangeIndex: 45222 entries, 0 to 45221
Data columns (total 14 columns):
age                45222 non-null int64
workclass          45222 non-null object
education_level    45222 non-null object
education-num      45222 non-null float64
marital-status     45222 non-null object
occupation         45222 non-null object
relationship       45222 non-null object
race               45222 non-null object
sex                45222 non-null object
capital-gain       45222 non-null float64
capital-loss       45222 non-null float64
hours-per-week     45222 non-null float64
native-country     45222 non-null object
income             45222 non-null object
dtypes: float64(4), int64(1), object(9)
memory usage: 4.8+ MB
</code></pre></div>
<p dir="auto">And then I get 103 total features after one-hot encoding.</p>
<p dir="auto">My code:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from sklearn.grid_search import GridSearchCV
from sklearn.metrics import fbeta_score, make_scorer,accuracy_score


clf = GradientBoostingClassifier()

parameters = {'loss': ['deviance', 'exponential'],
             'warm_start':[True,False],
              'max_depth':[4,5,6,7], - with this setup went away for 
              'n_estimators': [100, 200,300]
             }
scorer = make_scorer(fbeta_score, beta=0.5)

grid_obj = GridSearchCV(clf,param_grid=parameters,scoring=scorer,n_jobs=-1,verbose=10 )

grid_fit = grid_obj.fit(X_train,y_train)

best_clf = grid_fit.best_estimator_"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">grid_search</span> <span class="pl-k">import</span> <span class="pl-v">GridSearchCV</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">metrics</span> <span class="pl-k">import</span> <span class="pl-s1">fbeta_score</span>, <span class="pl-s1">make_scorer</span>,<span class="pl-s1">accuracy_score</span>


<span class="pl-s1">clf</span> <span class="pl-c1">=</span> <span class="pl-v">GradientBoostingClassifier</span>()

<span class="pl-s1">parameters</span> <span class="pl-c1">=</span> {<span class="pl-s">'loss'</span>: [<span class="pl-s">'deviance'</span>, <span class="pl-s">'exponential'</span>],
             <span class="pl-s">'warm_start'</span>:[<span class="pl-c1">True</span>,<span class="pl-c1">False</span>],
              <span class="pl-s">'max_depth'</span>:[<span class="pl-c1">4</span>,<span class="pl-c1">5</span>,<span class="pl-c1">6</span>,<span class="pl-c1">7</span>], <span class="pl-c1">-</span> <span class="pl-s1">with</span> <span class="pl-s1">this</span> <span class="pl-s1">setup</span> <span class="pl-s1">went</span> <span class="pl-s1">away</span> <span class="pl-s1">for</span> 
              '<span class="pl-s1">n_estimators</span>': [<span class="pl-c1">100</span>, <span class="pl-c1">200</span>,<span class="pl-c1">300</span>]
             }
<span class="pl-s1">scorer</span> <span class="pl-c1">=</span> <span class="pl-en">make_scorer</span>(<span class="pl-s1">fbeta_score</span>, <span class="pl-s1">beta</span><span class="pl-c1">=</span><span class="pl-c1">0.5</span>)

<span class="pl-s1">grid_obj</span> <span class="pl-c1">=</span> <span class="pl-v">GridSearchCV</span>(<span class="pl-s1">clf</span>,<span class="pl-s1">param_grid</span><span class="pl-c1">=</span><span class="pl-s1">parameters</span>,<span class="pl-s1">scoring</span><span class="pl-c1">=</span><span class="pl-s1">scorer</span>,<span class="pl-s1">n_jobs</span><span class="pl-c1">=</span><span class="pl-c1">-</span><span class="pl-c1">1</span>,<span class="pl-s1">verbose</span><span class="pl-c1">=</span><span class="pl-c1">10</span> )

<span class="pl-s1">grid_fit</span> <span class="pl-c1">=</span> <span class="pl-s1">grid_obj</span>.<span class="pl-en">fit</span>(<span class="pl-v">X_train</span>,<span class="pl-s1">y_train</span>)

<span class="pl-s1">best_clf</span> <span class="pl-c1">=</span> <span class="pl-s1">grid_fit</span>.<span class="pl-s1">best_estimator_</span></pre></div>
<h4 dir="auto">Versions</h4>

<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Darwin-16.7.0-x86_64-i386-64bit
('Python', '2.7.10 (default, Sep 23 2015, 04:34:14) \n[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]')
('NumPy', '1.13.3')
('SciPy', '1.0.0')
('Scikit-Learn', '0.19.1')
('Scikit-Learn', '0.19.1')"><pre class="notranslate"><code class="notranslate">Darwin-16.7.0-x86_64-i386-64bit
('Python', '2.7.10 (default, Sep 23 2015, 04:34:14) \n[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]')
('NumPy', '1.13.3')
('SciPy', '1.0.0')
('Scikit-Learn', '0.19.1')
('Scikit-Learn', '0.19.1')
</code></pre></div>
<h4 dir="auto">Actual Results</h4>

<p dir="auto">The output of incomplete process after 3 hours of running:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Fitting 3 folds for each of 48 candidates, totalling 144 fits
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=4 ...
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=4 ...
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=4 ...
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=4 ..
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=4 ..
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=4 ..
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=4 ...
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=4 ...
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=4, score=0.748376 - 1.1min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=4 ...
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=4, score=0.744681 - 1.1min
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=4, score=0.748376 - 1.1min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=4 ..
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=4 ..
[Parallel(n_jobs=-1)]: Done   2 tasks      | elapsed:  1.1min
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=4, score=0.755791 - 1.1min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=4 ..
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=4, score=0.744681 - 1.1min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=4 ...
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=4, score=0.755791 - 1.1min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=4 ...
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=4, score=0.755238 - 2.0min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=4 ...
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=4, score=0.748726 - 2.0min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=4 ..
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=4, score=0.755390 - 2.3min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=4 ..
[Parallel(n_jobs=-1)]: Done   9 tasks      | elapsed:  3.4min
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=4, score=0.748651 - 2.3min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=4 ..
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=4, score=0.757694 - 2.3min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=4, score=0.757694 - 2.3min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=4, score=0.753342 - 3.1min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=4, score=0.749265 - 3.2min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=4, score=0.753419 - 3.2min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=5, score=0.757060 - 1.8min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=5 ..
[Parallel(n_jobs=-1)]: Done  16 tasks      | elapsed:  5.2min
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=5, score=0.749191 - 1.8min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=4, score=0.759966 - 3.4min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=5, score=0.759275 - 1.9min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=5, score=0.758360 - 1.9min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=4, score=0.750019 - 3.2min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=4, score=0.759813 - 3.3min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=5, score=0.748366 - 1.8min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=5, score=0.759275 - 1.8min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=5, score=0.752455 - 3.0min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=5 ...
[Parallel(n_jobs=-1)]: Done  25 tasks      | elapsed:  8.3min
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=5, score=0.749569 - 3.0min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=5, score=0.752148 - 3.0min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=5, score=0.757131 - 3.1min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=5, score=0.749570 - 2.9min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=5, score=0.757797 - 2.9min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=5, score=0.747161 - 3.8min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=5, score=0.747114 - 3.8min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=6, score=0.752622 - 1.8min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=6, score=0.747398 - 1.8min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=6 ..
[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed: 11.4min
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=5, score=0.757843 - 3.6min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=5, score=0.747010 - 3.5min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=5, score=0.748530 - 3.3min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=5, score=0.757692 - 3.3min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=6, score=0.757374 - 1.8min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=6, score=0.752622 - 1.8min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=6, score=0.750770 - 1.8min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=6, score=0.757374 - 1.9min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=6, score=0.749492 - 4.6min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=6, score=0.745467 - 4.7min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=6, score=0.756200 - 5.1min
[Parallel(n_jobs=-1)]: Done  45 tasks      | elapsed: 17.6min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=6, score=0.752206 - 5.1min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=6, score=0.745467 - 5.2min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=6, score=0.755925 - 5.2min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=6, score=0.749421 - 7.4min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=6, score=0.744302 - 7.6min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=7 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=7, score=0.753965 - 3.7min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=7 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=7, score=0.742774 - 3.7min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=7 ..
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=6, score=0.749497 - 7.0min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=6, score=0.750551 - 7.2min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=7, score=0.755611 - 3.9min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=6, score=0.740919 - 7.2min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=7 ..
[Parallel(n_jobs=-1)]: Done  56 tasks      | elapsed: 24.8min
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=7, score=0.753425 - 3.9min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=7 ..
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=6, score=0.751236 - 7.2min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=7 ..
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=7, score=0.746959 - 4.1min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=7, score=0.754571 - 4.3min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=7 ..."><pre class="notranslate"><code class="notranslate">Fitting 3 folds for each of 48 candidates, totalling 144 fits
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=4 ...
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=4 ...
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=4 ...
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=4 ..
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=4 ..
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=4 ..
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=4 ...
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=4 ...
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=4, score=0.748376 - 1.1min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=4 ...
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=4, score=0.744681 - 1.1min
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=4, score=0.748376 - 1.1min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=4 ..
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=4 ..
[Parallel(n_jobs=-1)]: Done   2 tasks      | elapsed:  1.1min
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=4, score=0.755791 - 1.1min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=4 ..
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=4, score=0.744681 - 1.1min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=4 ...
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=4, score=0.755791 - 1.1min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=4 ...
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=4, score=0.755238 - 2.0min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=4 ...
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=4, score=0.748726 - 2.0min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=4 ..
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=4, score=0.755390 - 2.3min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=4 ..
[Parallel(n_jobs=-1)]: Done   9 tasks      | elapsed:  3.4min
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=4, score=0.748651 - 2.3min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=4 ..
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=4, score=0.757694 - 2.3min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=4, score=0.757694 - 2.3min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=4, score=0.753342 - 3.1min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=4, score=0.749265 - 3.2min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=4, score=0.753419 - 3.2min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=5, score=0.757060 - 1.8min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=5 ..
[Parallel(n_jobs=-1)]: Done  16 tasks      | elapsed:  5.2min
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=5, score=0.749191 - 1.8min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=4, score=0.759966 - 3.4min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=5, score=0.759275 - 1.9min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=5, score=0.758360 - 1.9min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=4, score=0.750019 - 3.2min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=4, score=0.759813 - 3.3min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=5, score=0.748366 - 1.8min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=5, score=0.759275 - 1.8min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=5 ...
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=5, score=0.752455 - 3.0min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=5 ...
[Parallel(n_jobs=-1)]: Done  25 tasks      | elapsed:  8.3min
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=5, score=0.749569 - 3.0min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=5, score=0.752148 - 3.0min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=5, score=0.757131 - 3.1min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=5 ..
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=5, score=0.749570 - 2.9min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=5, score=0.757797 - 2.9min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=5, score=0.747161 - 3.8min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=5, score=0.747114 - 3.8min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=6, score=0.752622 - 1.8min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=6, score=0.747398 - 1.8min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=6 ..
[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed: 11.4min
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=5, score=0.757843 - 3.6min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=5, score=0.747010 - 3.5min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=5, score=0.748530 - 3.3min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=5, score=0.757692 - 3.3min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=6, score=0.757374 - 1.8min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=6, score=0.752622 - 1.8min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=6, score=0.750770 - 1.8min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=6, score=0.757374 - 1.9min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=6, score=0.749492 - 4.6min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=6 ...
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=6, score=0.745467 - 4.7min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=200, loss=deviance, warm_start=True, max_depth=6, score=0.756200 - 5.1min
[Parallel(n_jobs=-1)]: Done  45 tasks      | elapsed: 17.6min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=6, score=0.752206 - 5.1min
[CV] n_estimators=300, loss=deviance, warm_start=False, max_depth=6 ..
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=6, score=0.745467 - 5.2min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=200, loss=deviance, warm_start=False, max_depth=6, score=0.755925 - 5.2min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=6, score=0.749421 - 7.4min
[CV] n_estimators=100, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=6, score=0.744302 - 7.6min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=7 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=7, score=0.753965 - 3.7min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=7 ..
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=7, score=0.742774 - 3.7min
[CV] n_estimators=100, loss=deviance, warm_start=False, max_depth=7 ..
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=6, score=0.749497 - 7.0min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=300, loss=deviance, warm_start=True, max_depth=6, score=0.750551 - 7.2min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=100, loss=deviance, warm_start=True, max_depth=7, score=0.755611 - 3.9min
[CV] n_estimators=200, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=6, score=0.740919 - 7.2min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=7 ..
[Parallel(n_jobs=-1)]: Done  56 tasks      | elapsed: 24.8min
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=7, score=0.753425 - 3.9min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=7 ..
[CV]  n_estimators=300, loss=deviance, warm_start=False, max_depth=6, score=0.751236 - 7.2min
[CV] n_estimators=200, loss=deviance, warm_start=False, max_depth=7 ..
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=7, score=0.746959 - 4.1min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=7 ...
[CV]  n_estimators=100, loss=deviance, warm_start=False, max_depth=7, score=0.754571 - 4.3min
[CV] n_estimators=300, loss=deviance, warm_start=True, max_depth=7 ...
</code></pre></div>

<h4 dir="auto">Expected Results</h4>
<p dir="auto">The below is the source code:</p>
<p dir="auto">import sqlite3<br>
import re<br>
import time<br>
import csv<br>
import numpy as np<br>
from sklearn.naive_bayes import MultinomialNB<br>
from sklearn.naive_bayes import BernoulliNB<br>
from sklearn.naive_bayes import GaussianNB<br>
from sklearn.utils import shuffle<br>
from sklearn.model_selection import train_test_split<br>
from sklearn.model_selection import cross_val_score<br>
from sklearn.learning_curve import learning_curve<br>
from sklearn.model_selection import GridSearchCV,RandomizedSearchCV<br>
from matplotlib import pyplot as pl<br>
from matplotlib.backends.backend_pdf import PdfPages<br>
from scipy import sparse<br>
from sklearn.feature_extraction.text import TfidfVectorizer<br>
from sklearn.feature_extraction.text import HashingVectorizer<br>
from sklearn.ensemble import RandomForestClassifier<br>
from sklearn.ensemble import ExtraTreesClassifier<br>
from sklearn.ensemble import AdaBoostClassifier<br>
from sklearn.tree import DecisionTreeClassifier<br>
from sklearn.datasets import make_blobs<br>
from sklearn import svm<br>
from sklearn.externals import joblib<br>
from sklearn.metrics import precision_recall_fscore_support<br>
from sklearn.metrics import f1_score<br>
from sklearn.metrics import accuracy_score<br>
import glob,os<br>
#MultiNomial</p>
<p dir="auto">from xgboost import XGBClassifier<br>
import xgboost<br>
from tpot import TPOTClassifier</p>
<p dir="auto">tpot_config = {<br>
'xgboost.XGBClassifier': {<br>
'n_estimators':[50,100,150,200],<br>
'max_depth':[2,3,4,5,6,7,8,9],<br>
'min_child_weight':[2,3,4,5],<br>
'colsample_bytree':[0.2,0.6,0.8],<br>
'colsample_bylevel':[0.2,0.6,0.8],<br>
'objective':['multi:softprob']</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="}"><pre class="notranslate"><code class="notranslate">}
</code></pre></div>
<p dir="auto">}</p>
<p dir="auto">param = {<br>
'n_estimators':[200],<br>
'max_depth':[3,5,4],<br>
'min_child_weight':[4,5,3],<br>
'colsample_bytree':[0.2,0.6,0.8],<br>
'colsample_bylevel':[0.2,0.6,0.8]</p>
<p dir="auto">}<br>
#pipeline_optimizer = TPOTClassifier(generations=5, population_size=20, cv=2,max_eval_time_mins=30,random_state=42, verbosity=200, config_dict=tpot_config,n_jobs = 10)</p>
<p dir="auto">clf=XGBClassifier(objective= 'multi:softprob')</p>
<p dir="auto">gsearch1 = GridSearchCV(estimator = clf, param_grid = param,cv=2,verbose = 100,n_jobs = -1)<br>
gsearch1.fit(X_train, y_train)<br>
print gsearch1.bestscore<br>
print gsearch1.bestparams</p>
<p dir="auto">#Omitted code relevant to data loading. It works fine for n_jobs =1</p>
<p dir="auto">I am using Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 26 2016, 12:10:39).<br>
scikit-learn (0.19.1)</p>
<p dir="auto">Because of this issue, TPOT also hangs as it uses SciKit learn GridSearchCV for internal operations.</p>