proofwe have "   gt (xxy [ xye[gt (xxyy ( where the inequality follows from eu (eu) for any (conditionalexpectation consequentlyby the tower property  ii  gt (xe gt (xxy ( bagged estimator unfortunatelymultiple independent data sets are rarely available but we can substitute them by bootstrapped ones specificallyinstead of the tb setswe can obtain random training sets tbby resampling them from single (fixedtraining set tsimilar to algorithm and use them to train separate models by model averaging as in ( we obtain the bootstrapped aggregated estimator or bagged estimator of the formb gbag (xgt ( ( = algorithm bootstrap aggregation sampling inputtraining set {(xi yi )}ni= and resample size outputbootstrapped data sets for to do tb for to do draw ( dnue /select random index tbtb{(xi yi ) return tbb remark (bootstrap aggregation for classification problemsnote that ( is suitable for handling regression problems howeverthe bagging idea can be readily extended to handle classification settings as well for examplegbag can take the majority vote among {gtb} bthat isto accept the most frequent class among predictors while bagging can be applied for any statistical model (such as decision treesneural networkslinear regressionk-nearest neighborsand so on)it is most effective for predictors that are sensitive to small changes in the training set the reason becomes clear when we decompose the expected generalization risk as `(gt ` ( [gt (xxg( )) [var[gt (xx]]{ { expected squared bias expected variance (