Gradient Boosting. This is a boosting algorithm of an ensemble method combining several weak learners into a strong learner training predictor sequentially. Gradient boosting builds many small models (e.g., decision trees) to sequentially reduce the residual errors from the previous steps by employing a gradient descent optimization algorithm. AdaBoost (Adaptative Boosting) Regressor. Similarly to Gradient Boosting, AdaBoost is a boosting method because it combines sequentially weak models However, in contrast to Gradient Boosting, it adjusts the instance weights at every interaction. In other words, the predictors are trained sequentially, each one trying to correct the predecessors' output. In this kind of algorithm, a component of randomness is removed compared to other techniques, considering that it uses the previous predictor to calibrate the next one.