we explain the concepts and terminologies related to decision trees using an example suppose we have hypothetical dataset of car models from different manufacturers assume each data point has features like fuel_capacityengine_capacitypriceyear_of_purchasemiles_drivenand mileage given this datawe need model that can predict the mileage given other attributes since decision trees are supervised learning algorithmswe have certain number of data points with actual mileage values available decision tree starts off at the root and divides the dataset into two or more non-overlapping subsetseach represented as child node of the root it divides the root into subsets based on specific attribute it goes on performing the split at every node until leaf node is achieved where the target value is available there might be lot many questions on how it all happensand we will get to each of them in bit for better understandingassume figure - is the structure of the decision tree inferred from the dataset at hand year of purchase >= miles driven price > cc cc mi/gal fuel capacity mi/gal engine capacity mi/gal > miles miles >$ $ gal mi/gal mi/gal > gal mi/gal figure - sample decision tree the visualization depicted in figure - showcases sample decision tree with leaf nodes pointing toward target values the tree starts off by splitting the dataset at root based on year of purchase with left child representing purchases before and right child for purchases after and similarly for other nodes when presented with new/unseen data pointwe simply traverse the tree and arrive at leaf node which determines the target value even though the previous example is simpleit clearly brings out the interpretability of the model as well as its ability to learn simple rules ode splitting decision trees work in top-down manner and node splitting is an important concept for any decision tree algorithm most algorithms follow greedy approach to divide the input space into subsets the basic process in simple terms is to try and split data points using different attributes/features and test against cost function the split resulting in least cost is selected at every step classification and regression problems use different set of cost functions some of the most common ones are as follows mean squared error (mse)used mainly for regression treesit is calculated as the square of difference between observed and predicted values