further reading breiman' book on decision trees[ ]serves as great starting point some additional advances can be found in [ from the computational point of viewthere exists an efficient recursive procedure for tree pruningsee and in [ several advantages and disadvantages of using decision trees are debated in [ detailed discussion on bagging and random forests can be found in [ and [ ]respectively freund and schapire [ provide the first boosting algorithmthe adaboost while adaboost was developed in the context of the computational complexity of learningit was later discovered by friedman [ that adaboost is special case of an additive model in additionit was shown that for any differentiable loss functionthere exists an efficient boosting procedure which mimics the gradient descent algorithm the foundation of the resulting gradient boosting method is detailed in [ python packages that implement gradient boosting include xgboost and lightgbm exercises show that any training set {(xyi ) ncan be fitted via tree with zero training loss suppose during the construction of decision tree we wish to specify constant regional prediction function gw on the region rw based on the training data in rw say {( )(xk yk )show that gw ( : - ki= yi minimizes the squared-error loss using the program from section write basic implementation of decision tree for binary classification problem implement the misclassificationgini indexand entropy impurity criteria to split nodes compare the results suppose in the decision tree of example there are blue and red data points in certain tree region calculate the misclassification impuritythe gini impurityand the entropy impurity repeat these calculations for blue and red data points consider the procedure of finding the best splitting rule for categorical variable with labels from section show that one needs to consider subsets of { kto find the optimal partition of labels reproduce figure using the following classification data from sklearn datasets import make_blobs xy make_blobs n_samples = n_features = centers = random_state = cluster_std = prove ( )that isshow that { loss( gw ( ) (gi ww =