train cv loss tree depth figure the cross-validation and the training loss as function of the tree depth for binary classification problem smaller than according to the proposed procedurethis node will not be split further this mayhoweverbe sub-optimalbecause it could happen that one of the node' descendantsif splitcould lead to major decrease in loss to address these issuesa so-called pruning routine can be employed the idea is as follows we first grow very deep tree and then prune (remove nodesit upwards until we reach the root node consequentlythe pruning process causes the number of tree nodes to decrease while the tree is being prunedthe generalization risk gradually decreases up to the point where it starts increasing againat which point the pruning is stopped this decreasing/increasing behavior is due to the bias-variance tradeoff ( tree pruning we next describe the details to start withlet and be tree nodes we say that is descendant of if there is path down the treewhich leads from to if such path existswe also say that is an ancestor of consider the tree in figure to formally define pruningwe will require the following definition an example of pruning is demonstrated in figure definition branches and pruning tree branch tv of the tree is sub-tree of rooted at node the pruning of branch tv from tree is performed via deletion of the entire branch tv from except the branch' root node the resulting pruned tree is denoted by tv sub-tree tv is called pruned sub-tree of we indicate this with the notation tv or tv basic decision tree pruning procedure is summarized in algorithm tree branch