Making scikit-learn train on all training data after cross-validation
My recommendation is to not use the cross-validation split that had the best performance. That could potential give you problems with high bias. Afterall, the performance just happened to be good because there was a fold used for testing that just happened to match the data used for training. When you generalize it to the real world, that probably won't happen.
A strategy I got from Andrew Ng is to have a train, dev, and test sets. I would first split your dataset into a test and train set. Then use cross fold validation on your training set, where effectively the training set will be split into training and dev sets. Do cross fold validation to validate your model and store the precision and recall and other metrics to build a ROC curve. Average the values and report those. You can also tune the hyperparameters using your dev set as well.
Next, train the model with the entire training set, then validate the model with your hold out test set.
I'm using scikit-learn to train classifiers. I want also to do cross validation, but after cross-validation I want to train on the entire dataset. I found that cross_validation.cross_val_score() just returns the scores.
Edit: I would like to train the classifier that had the best cross-validation score with all of my data.