title: Sklearn: Evaluate performance of each classifier of OneVsRestClassifier inside GridSearchCV
body: <p>I am dealing with multi-label classification with <code>OneVsRestClassifier</code> and <code>SVC</code>, </p>
  
  <pre><code>from sklearn.datasets import make_multilabel_classification
  from sklearn.multiclass import OneVsRestClassifier
  from sklearn.svm import SVC
  from sklearn.grid_search import GridSearchCV
  
  L=3
  X, y = make_multilabel_classification(n_classes=L, n_labels=2,
                                    allow_unlabeled=True,
                                    random_state=1, return_indicator=True)    
  model_to_set = OneVsRestClassifier(SVC())
  
  parameters = {
      "estimator__C": [1,2,4,8],
      "estimator__kernel": ["poly","rbf"],
      "estimator__degree":[1, 2, 3, 4],
  }
  
  model_tunning = GridSearchCV(model_to_set, param_grid=parameters,
                               scoring='f1')
  
  model_tunning.fit(X, y)
  
  print model_tunning.best_score_
  print model_tunning.best_params_
  
  #0.855175822314
  #{'estimator__kernel': 'poly', 'estimator__C': 1, 'estimator__degree': 3}
  </code></pre>
  
  <p><strong>1st question</strong> </p>
  
  <p>What is the number <code>0.85</code> representing for? Is it the best score among the <code>L</code> classifiers or the averaged one? Similarly, does the set of parameters stand for the best-scorer among <code>L</code> classifiers? </p>
  
  <p><strong>2nd question</strong></p>
  
  <p>Based on the fact that, if I am right, the <code>OneVsRestClassifier</code> literally builds <code>L</code> classifiers for each label, one can expect to access or observe the performance of EACH LABEL. But how, in the above example, to obtain <code>L</code> scores from the <code>GridSearchCV</code> object?</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>To simplify the problem and help myself understand more about <code>OneVsRestClassifier</code>, before tuning model, </p>
  
  <pre><code>model_to_set.fit(X,y)
  gp = model_to_set.predict(X) # the "global" prediction
  fp = model_to_set.estimators_[0].predict(X) # the first-class prediction
  sp = model_to_set.estimators_[1].predict(X) # the second-class prediction
  tp = model_to_set.estimators_[2].predict(X) # the third-class prediction
  </code></pre>
  
  <p>It can be shown that <code>gp.T[0]==fp</code>, <code>gp.T[1]==sp</code> and <code>gp.T[2]==tp</code>. So the "global" prediction is simply the 'sequential' <code>L</code> individual predictions and the <strong>2nd question is solved</strong>.</p>
  
  <p>But it is still confusing for me that if one meta-classifier <code>OneVsRestClassifier</code> contains <code>L</code> classifiers, how can <code>GridSearchCV</code> returns only ONE best score, corresponding to one of 4*2*4 sets of parameters, for a meta-classifier <code>OneVsRestClassifier</code> having <code>L</code> classifiers? </p>
  
  <p>It would be fairly appreciated to see any comment.</p>