title: How to interpret best score from GridSearch?
body: <p>I train different classifiers with different set of data and I need to understand how can I properly measure effectiveness of classifier.</p>
  
  <p>Here's my code:</p>
  
  
  
  <pre class="lang-python prettyprint-override"><code>iris = load_iris()
  
  param_grid = {
      'criterion': ['gini', 'entropy'],
      'max_depth': np.arange(4, 6)
  }
  
  tree = GridSearchCV(DecisionTreeClassifier(), param_grid)
  X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target)
  
  tree.fit(X_train, y_train)
  tree_preds = tree.predict(X_test)
  tree_performance = accuracy_score(y_test, tree_preds)
  
  print 'Best params: ', tree.best_params_
  print 'Best score: ', tree.best_score_
  print 'DecisionTree score: ', tree_performance
  </code></pre>
  
  <p>Question is, what actually is a best score from GridSearchCV? And how it differs from results used in <code>accuray_score</code> function?</p>
  
  <p>As I understand, <code>accuracy_score</code> takes classes of test set and compares it with results predicted by algorithm. The result is the percentage of properly classified items. But what is <code>best_score_</code>?</p>
  
  <p>These two values are different and an example output from my script looks like this:</p>
  
  <pre class="lang-python prettyprint-override"><code>Best score:  0.955357142857
  DecisionTree score:  0.947368421053
  </code></pre>