title: How to estimate the progress of a GridSearchCV from verbose output in Scikit-Learn?
body: <p>Right now I'm running a pretty aggressive grid search.  I have <code>n=135 samples</code> and I am running <code>23 folds</code> using a custom cross-validation train/test list. I have my <code>verbose=2</code>. </p>
  
  <p>The following is what I ran:</p>
  
  <pre><code>param_test = {"loss":["deviance"],
             'learning_rate':[0.01, 0.025, 0.05, 0.075, 0.1, 0.15, 0.2],
             "min_samples_split": np.linspace(0.1, 0.5, 12),
             "min_samples_leaf": np.linspace(0.1, 0.5, 12),
             "max_depth":[3,5,8],
            "max_features":["log2","sqrt"],
            "min_impurity_split":[5e-6, 1e-7, 5e-7],
            "criterion": ["friedman_mse",  "mae"],
             "subsample":[0.5, 0.618, 0.8, 0.85, 0.9, 0.95, 1.0],
            "n_estimators":[10]}
  
  Mod_gsearch = GridSearchCV(estimator = GradientBoostingClassifier(),
                             param_grid = param_test, scoring="accuracy",n_jobs=32, iid=False, cv=cv_indices, verbose=2)
  </code></pre>
  
  <p>I took a look at the verbose output in <code>stdout</code>:</p>
  
  <pre><code>$head gridsearch.o8475533
  Fitting 23 folds for each of 254016 candidates, totalling 5842368 fits
  </code></pre>
  
  <p>Based on this, it looks like there are <code>5842368</code> permutations of cross-validation pairs using my grid params.</p>
  
  <pre><code>$ grep -c  "[CV]" gridsearch.o8475533
  7047332 
  </code></pre>
  
  <p>It looks like there are around 7 million cross-validations that have been done so far but that's more than the <code>5842368</code> total fits...</p>
  
  <pre><code>7047332/5842368 = 1.2062458236
  </code></pre>
  
  <p>Then when I look at the <code>stderr</code> file: </p>
  
  <pre><code>$ cat ./gridsearch.e8475533
  [Parallel(n_jobs=32)]: Done 132 tasks      | elapsed:    1.2s
  [Parallel(n_jobs=32)]: Done 538 tasks      | elapsed:    2.8s
  [Parallel(n_jobs=32)]: Done 1104 tasks      | elapsed:    4.8s
  [Parallel(n_jobs=32)]: Done 1834 tasks      | elapsed:    7.9s
  [Parallel(n_jobs=32)]: Done 2724 tasks      | elapsed:   11.6s
  ...
  [Parallel(n_jobs=32)]: Done 3396203 tasks      | elapsed: 250.2min
  [Parallel(n_jobs=32)]: Done 3420769 tasks      | elapsed: 276.5min
  [Parallel(n_jobs=32)]: Done 3447309 tasks      | elapsed: 279.3min
  [Parallel(n_jobs=32)]: Done 3484240 tasks      | elapsed: 282.3min
  [Parallel(n_jobs=32)]: Done 3523550 tasks      | elapsed: 285.3min
  </code></pre>
  
  <p><em>My goal:</em></p>
  
  <p><strong>How can I know the progress of my gridsearch with respect to the total time it may take?</strong> </p>
  
  <p><em>What I'm confused about:</em></p>
  
  <p><strong>What is the relationship between <code>[CV]</code> lines in <code>stdout</code>, total # of fits in <code>stdout</code>, and tasks in <code>stderr</code>?</strong> </p>