context known as weak learners)like shallow trees each tree can only provide good predictions on part of the dataand so more and more trees are added to iteratively improve performance gradient boosted trees are frequently the winning entries in machine learning competitionsand are widely used in industry they are generally bit more sensitive to parameter settings than random forestsbut can provide better accuracy if the parameters are set correctly apart from the pre-pruning and the number of trees in the ensembleanother important parameter of gradient boosting is the learning_ratewhich controls how strongly each tree tries to correct the mistakes of the previous trees higher learning rate means each tree can make stronger correctionsallowing for more complex models adding more trees to the ensemblewhich can be accomplished by increasing n_estimatorsalso increases the model complexityas the model has more chances to correct mistakes on the training set here is an example of using gradientboostingclassifier on the breast cancer dataset by default trees of maximum depth and learning rate of are usedin[ ]from sklearn ensemble import gradientboostingclassifier x_trainx_testy_trainy_test train_test_splitcancer datacancer targetrandom_state= gbrt gradientboostingclassifier(random_state= gbrt fit(x_trainy_trainprint("accuracy on training set{ }format(gbrt score(x_trainy_train))print("accuracy on test set{ }format(gbrt score(x_testy_test))out[ ]accuracy on training set accuracy on test set as the training set accuracy is %we are likely to be overfitting to reduce overfittingwe could either apply stronger pre-pruning by limiting the maximum depth or lower the learning ratesupervised machine learning algorithms