GridSearchCV with Random Forest Classifier I'm working with a supervised learning problem and trying to predict a binary label and using a Random Forest to do so.
I'm trying to tune my hyper-parameters to give me a best model based on my data.
I can do this with GridSearchCV(), but is this correct to do with a random forest?
If I'm using GridSearchCV(), the training set and testing set change with each fold. From my understanding we can we set oob_true = True in RandomForestClassifier(), we are already evaluating on the out-of-bag samples (so CV is kind of already built in RF).
What is the convention to hyper-parameter tune with Random Forest to get the best OOB score in sklearn? Can I just loop through a set of parameters and fit on the same training and testing set? Can I use GridSearchCV(), or does that make no sense with RF?
You can definitely use GridSearchCV with Random Forest. In fact you should use GridSearchCV to find the best parameters that will make your oob_score very high.
Some parameters to tune are:
n_estimators: Number of tree your random forest should have. The more n_estimators the less overfitting. You should try from 100 to 5000 range.
max_depth: max_depth of each tree. You should specify certain max_depth so that your model don't memorise train examples.
min_sample_split: the minimum number of samples to have before splitting into new nodes.
and many more...
But these are the main hyperparameters we tune to get a forest that works well.And to get these hyperparams you should use GridSeachCV.