cross-entropy gini index misclassification impurity figure entropyginiand misclassification impurities for binary classificationwith class frequencies and the entropy impurity was normalized (divided by )to ensure that all impurity measures attain the same maximum value of / at / significant advantagein terms of training lossto split regions ultimatelythe quality of tree is determined by its predictive performance (generalization riskand the termination condition should aim to strike balance between minimizing the approximation error and minimizing the statistical erroras discussed in section example (fixed tree depthto illustrate how the tree depth impacts on the generalization riskconsider figure which shows the typical behavior of the cross-validation loss as function of the tree depth recall that the cross-validation loss is an estimate of the expected generalization risk complicated (deeptrees tend to overfit the training data by producing many divisions of the feature space as we have seenthis overfitting problem is typical of all learning methodssee and in particular example to concludeincreasing the maximal depth does not necessarily result in better performance loss tree depth figure the ten-fold cross-validation loss as function of the maximal tree depth for classification problem the optimal maximal tree depth is here