fit to the breast cancer dataset we can see that the feature importances of the gradient boosted trees are somewhat similar to the feature importances of the random foreststhough the gradient boosting completely ignored some of the features as both gradient boosting and random forests perform well on similar kinds of dataa common approach is to first try random forestswhich work quite robustly if random forests work well but prediction time is at premiumor it is important to squeeze out the last percentage of accuracy from the machine learning modelmoving to gradient boosting often helps if you want to apply gradient boosting to large-scale problemit might be worth looking into the xgboost package and its python interfacewhich at the time of writing is faster (and sometimes easier to tunethan the scikit-learn implementation of gradient boosting on many datasets strengthsweaknessesand parameters gradient boosted decision trees are among the most powerful and widely used models for supervised learning their main drawback is that they require careful tuning of the parameters and may take long time to train similarly to other tree-based modelsthe algorithm works well without scaling and on mixture of binary and continuous features as with other tree-based modelsit also often does not work well on high-dimensional sparse data the main parameters of gradient boosted tree models are the number of treesn_esti matorsand the learning_ratewhich controls the degree to which each tree is allowed to correct the mistakes of the previous trees these two parameters are highly supervised machine learning algorithms