the major idea of random forests is to perform bagging in combination with "decorrelationof the trees by including only subset of features during the tree construction for each bootstrapped training set tbwe build decision tree using randomly selected subset of features for the splitting rules this simple but powerful idea will decorrelate the treessince strong predictors will have smaller chance to be considered at the root levels consequentiallywe can expect to improve the predictive performance of the bagged estimator the resulting predictor (random forestconstruction is summarized in algorithm algorithm random forest construction inputtraining set {(xi yi )}ni= the number of trees in the forest band the number of features to be includedwhere is the total number of features in outputensemble of trees generate bootstrapped training sets { via algorithm for to do train decision tree gtbvia algorithm where each split is performed using randomly selected features out of return {gt } = for regression problemsthe output of algorithm is combined to yield the random forest prediction functionb gt (xgrf (xb = in the classification settingsimilar to remark we take instead the majority vote from the {gtbexample (random forest for regression treewe continue with the basic bagging example for regression treein which we compared the decision tree estimator with the corresponding bagged estimator herehoweverwe use the random forest with trees and subset size it can be seen that the random forest' score is outperforming that of the bagged estimator baggingexamplerf py from from from from sklearn datasets import make_friedman sklearn model_selection import train_test_split sklearn metrics import _score sklearn ensemble import randomforestregressor create regression problem n_points points xy make_friedman n_samples =n_points n_features = noise = random_state = split to train /test set x_train x_test y_train y_test train_test_split (xytest_size = random_state = rf randomforestregressor n_estimators = oob_score true