ting decision trees have two advantages over many of the algorithms we've discussed so farthe resulting model can easily be visualized and understood by nonexperts (at least for smaller trees)and the algorithms are completely invariant to scaling of the data as each feature is processed separatelyand the possible splits of the data don' depend on scalingno preprocessing like normalization or standardization of features is needed for decision tree algorithms in particulardecision trees work well when you have features that are on completely different scalesor mix of binary and continuous features the main downside of decision trees is that even with the use of pre-pruningthey tend to overfit and provide poor generalization performance thereforein most applicationsthe ensemble methods we discuss next are usually used in place of single decision tree ensembles of decision trees ensembles are methods that combine multiple machine learning models to create more powerful models there are many models in the machine learning literature that belong to this categorybut there are two ensemble models that have proven to be effective on wide range of datasets for classification and regressionboth of which use decision trees as their building blocksrandom forests and gradient boosted decision trees random forests as we just observeda main drawback of decision trees is that they tend to overfit the training data random forests are one way to address this problem random forest is essentially collection of decision treeswhere each tree is slightly different from the others the idea behind random forests is that each tree might do relatively good job of predictingbut will likely overfit on part of the data if we build many treesall of which work well and overfit in different wayswe can reduce the amount of overfitting by averaging their results this reduction in overfittingwhile retaining the predictive power of the treescan be shown using rigorous mathematics to implement this strategywe need to build many decision trees each tree should do an acceptable job of predicting the targetand should also be different from the other trees random forests get their name from injecting randomness into the tree building to ensure each tree is different there are two ways in which the trees in random forest are randomizedby selecting the data points used to build tree and by selecting the features in each split test let' go into this process in more detail supervised machine learning algorithms