mean absolute errorused for regression treesit is similar to mse though we only use the difference between the observed and predicted values variance reductionthis was first introduced with cart algorithmand it uses the standard formula of variance and we choose the split that results in least variance gini impurity/indexmainly used by classification treesit is measure of randomly chosen data point to have an incorrect label given it was labeled randomly information gainagain used mainly for classification problemsit is also termed as entropy we choose the splits based on the amount of information gain the higher information gainthe better it is ##note these are some of the most commonly used cost functions there are many more which are used under specific scenarios topping criteria as mentioneddecision trees follow greedy recursive splitting of nodesbut how or when do they stopthere are many strategies applied to define the stopping criteria the most common being the minimum count of data points node splitting is stopped if further splitting would violate this constraint another constraint used is the depth of the tree the stopping criteria together with other parameters help us achieve trees that can generalize well tree that is very deep or has too many non-leaf nodes often results in overfitting hyperparameters hyperparameters are the knobs and controls we set with an aim to optimize the model' performance on unseen data these hyperparameters are different from parameters which are learned by our learning algorithm over the course of training process hyperparameters help us achieve objectives of avoiding overfitting and so on decision trees provide us with quite few hyperparameters to play withsome of which we discussed in maximum depthminimum samples for leaf nodesminimum samples to split internal nodesmaximum leaf nodesand so on are some of the hyperparameters actively used to improve performance of decision trees we will use techniques like grid search (refresh your memory from to identify optimal values for these hyperparameters in the coming sections decision tree algorithms decision trees have been around for quite some time now they have evolved with improvements in algorithms based on different techniques over the years some of the most commonly used algorithms are listed as followscart or classification and regression tree id or iterative dichotomizer now that we have decent understanding of decision treeslet' see if we can achieve improvements by using them for our regression problem of predicting the bike sharing demand