Obtain optimal number of boosting iterations in GradientBoostingClassifier using grid search
Currently there is no way to directly get the optimum number of estimators from GradientBoostingClassifier. If you also pass n_estimators in the parameter grid to GridSearchCV it will only try the exact values you give it, and return one of these.
We are looking to improve this, by searching over the number of estimators automatically.
With GradientBoostingClassifier suppose I set n_estimators to 2000 and use GridSearchCV to search across learning_rate in [0.01, 0.05, 0.10] - how do I know the number of boosting iterations that produced the optimal result - is the model always going to fit 2000 trees for each value of learning_rate or is it going to use the optimal number of boosting iterations for each of these values? It wouldn't make sense to also include n_estimators in the grid search and search for all values in [1,2000].