ecision rees and nsemble ethods statistical learning methods based on decision trees have gained tremendous popularity due to their simplicityintuitive representationand predictive accuracy this gives an introduction to the construction and use of such trees we also discuss two key ensemble methodsnamely bootstrap aggregation and boostingwhich can further improve the efficiency of decision trees and other learning methods introduction tree-based methods provide simpleintuitiveand powerful mechanism for both regression and classification the main idea is to divide (potentially complicatedfeature space into smaller regions and fit simple prediction function to each region for examplein regression settingone could take the mean of the training responses associated with the training features that fall in that specific region in the classification settinga commonly used prediction function takes the majority vote among the corresponding response variables we start with simple classification example example (decision tree for classificationthe left panel of figure shows training set of two-dimensional points (featuresfalling into two classes (red and bluehow should the new feature vector (black pointbe classified figure lefttraining data and new feature righta partition of the feature space