residuals yalpha *g_ list of basic regressor g_boost [for in range boostingrounds )h_i decisiontreeregressor max_depth = h_i fit(xresiduals residuals residuals alpha *h_i predict (xg_boost append (h_ireturn g_ g_boost def predict (g_ g_boost ,alpha )yhat alpha *g_ *np ones(len( )for in range (leng_boost ))yhat yhatalpha g_boost [jpredict (xreturn yhat np random seed ( sz create data set , make_regression n_samples =sz n_features = n_informative = noise = boosting algorithm boostingrounds alphas [ for alpha in alphas g_ g_boost trainboost (alpha boostingrounds , ,yyhat predict (g_ g_boost alpha xplot tmpx np reshape (np linspace - , , ,( , )yhatx predict (g_ g_boost alpha tmpxf plt figure (plt plot( , ,'*'plt plot(tmpx yhatx plt show (the parameter can be viewed as step size made in the direction of the negative gradient of the squared-error training loss to see thisnote that the negative gradient loss (yi (yi ) = (yi gb- (xi ) =gb- (xi =gb- (xi is two times the residual (bi given in ( that is used in algorithm to fit the prediction function hb in factone of the major advances in the theory of boosting was the recognition that one can use similar gradient descent method for any differentiable loss function the