7.13 Conclusion | When to use decision trees--[Machine Learning | Andrew Ng]
 Both decision trees, including tree ensembles, as well as neural networks, are very powerful, very effective learning algorithms. When should you pick one or the other? Let's look at some of the pros and cons of each. Decision trees and tree ensembles will often work well on tabular data, also called structured data. And what that means is if your data set looks like a giant spreadsheet, then decision trees would be worth considering. So for example, in the housing price prediction application, we had a data set with features corresponding to the size of the house, the number of bedrooms, the number of floors, and the age of the home. And that type of data stored in a spreadsheet with either categorical or continuous value features and both for classification or for regression tasks, where you're trying to predict a discrete category or predict a number. All of these problems are ones that decision trees can do well on. In contrast, I would not recommend using decision trees and tree ensembles on unstructured data. And that's data such as images, video, audio, and text that you're less likely to store in a spreadsheet format. Neural networks, as we'll see in a second, will tend to work better for unstructured data tasks. One huge advantage of decision trees and tree ensembles is that they can be very fast to train. You might remember this diagram from the previous week in which we talked about the iterative loop of machine learning development. If your model takes many hours to train, then that limits how quickly you can go through this loop and improve the performance of your algorithm. But because decision trees, including tree ensembles, tend to be pretty fast to train, that allows you to go through this loop more quickly and maybe more efficiently improve the performance of your learning algorithm. Finally, small decision trees may be human interpretable. If you are training just a single decision tree and if that decision tree has only, say, a few dozen nodes, you may be able to print out the decision tree to understand exactly how it's making decisions. I think that the interpretability of decision trees is sometimes a bit overstated because when you build an ensemble of a hundred trees and if each of those trees has hundreds of nodes, then looking at that ensemble to figure out what it's doing does become difficult and may need some separate visualization techniques. But if you have a small decision tree, you can actually look at it and see, oh, it's classifying whether something is a cat by looking at certain features in certain ways. If you've decided to use a decision tree or tree ensemble, I would probably use XGBoost for most of the applications I would work on. One slight downside of a tree ensemble is that it is a bit more expensive than a single decision tree. And so if you had a very, very constrained computational budget, you might use a single decision tree, but other than that setting, I would almost always use a tree ensemble and use XGBoost in particular. How about neural networks? In contrast to decision trees and tree ensembles, it works well on all types of data, including tabular or structured data, as well as unstructured data, as well as mixed data that includes both structured and unstructured components. Whereas on tabular structured data, neural networks and decision trees are often both competitive on unstructured data, such as images, video, audio, and text, a neural network will really be the preferred algorithm and not a decision tree or tree ensemble. On the downside, though, neural networks may be slower than a decision tree. A large neural network can just take a long time to train. Other benefits of neural networks includes that it works with transfer learning. And this is really important because for many applications where you have only a small data set, being able to use transfer learning and carry out pre-training on a much larger data set that is critical to getting competitive performance. Finally, there are also some technical reasons why it might be easier to string together multiple neural networks to build a larger machine learning system. And the basic reason is neural networks compute the output y as a smooth or continuous function of the input x. And so even if you string together a lot of different models, the outputs of all of these different models is self-differentiable. And so you can train them all at the same time using gradient descent. Whereas with a decision tree, you can only train one tree at a time. Finally, if you're building a system of multiple machine learning models working together, it might be easier to string together and train multiple neural networks than multiple decision trees. The reasons for this are quite technical, and you don't need to worry about it for the purpose of this course. But it relates to that even when you string together multiple neural networks, you can train them all together using gradient descent. Whereas for decision trees, you can only train one decision tree at a time. So that's it. You've reached the end of the videos for this course on advanced learning algorithms. Thank you for sticking with me all this way and congratulations on getting to the end of the videos on advanced learning algorithms. You've now learned how to build and use both neural networks and decision trees, and also heard about a variety of tips, practical advice on how to get these algorithms to work well for you. But even with all that you've seen on supervised learning, that's just part of what learning algorithms can do. Supervised learnings need label datasets with the labels Y on your training set. There's another set of very powerful algorithms called unsupervised learning algorithms, where you don't even need labels Y for the algorithm to figure out very interesting patterns and to do things with the data that you have. So I look forward to seeing you also in the third and final course of this specialization, which will be on unsupervised learning. Now before you finish up this course, I hope you also enjoy practicing the ideas of decision trees in their practice quizzes and in their practice labs. I'd like to wish you the best of luck in the practice labs. For those of you that may be Star Wars fans, let me say may the forest be with you.
[{"start": 0.0, "end": 8.0, "text": " Both decision trees, including tree ensembles, as well as neural networks, are very powerful,"}, {"start": 8.0, "end": 10.200000000000001, "text": " very effective learning algorithms."}, {"start": 10.200000000000001, "end": 12.68, "text": " When should you pick one or the other?"}, {"start": 12.68, "end": 15.64, "text": " Let's look at some of the pros and cons of each."}, {"start": 15.64, "end": 21.0, "text": " Decision trees and tree ensembles will often work well on tabular data, also called structured"}, {"start": 21.0, "end": 22.44, "text": " data."}, {"start": 22.44, "end": 28.080000000000002, "text": " And what that means is if your data set looks like a giant spreadsheet, then decision trees"}, {"start": 28.08, "end": 30.4, "text": " would be worth considering."}, {"start": 30.4, "end": 36.72, "text": " So for example, in the housing price prediction application, we had a data set with features"}, {"start": 36.72, "end": 41.08, "text": " corresponding to the size of the house, the number of bedrooms, the number of floors,"}, {"start": 41.08, "end": 42.8, "text": " and the age of the home."}, {"start": 42.8, "end": 49.08, "text": " And that type of data stored in a spreadsheet with either categorical or continuous value"}, {"start": 49.08, "end": 53.84, "text": " features and both for classification or for regression tasks, where you're trying to predict"}, {"start": 53.84, "end": 57.36, "text": " a discrete category or predict a number."}, {"start": 57.36, "end": 63.04, "text": " All of these problems are ones that decision trees can do well on."}, {"start": 63.04, "end": 69.6, "text": " In contrast, I would not recommend using decision trees and tree ensembles on unstructured data."}, {"start": 69.6, "end": 75.16, "text": " And that's data such as images, video, audio, and text that you're less likely to store"}, {"start": 75.16, "end": 77.8, "text": " in a spreadsheet format."}, {"start": 77.8, "end": 81.16, "text": " Neural networks, as we'll see in a second, will tend to work better for unstructured"}, {"start": 81.16, "end": 83.24, "text": " data tasks."}, {"start": 83.24, "end": 88.32, "text": " One huge advantage of decision trees and tree ensembles is that they can be very fast to"}, {"start": 88.32, "end": 90.28, "text": " train."}, {"start": 90.28, "end": 95.96, "text": " You might remember this diagram from the previous week in which we talked about the iterative"}, {"start": 95.96, "end": 98.72, "text": " loop of machine learning development."}, {"start": 98.72, "end": 104.11999999999999, "text": " If your model takes many hours to train, then that limits how quickly you can go through"}, {"start": 104.11999999999999, "end": 107.47999999999999, "text": " this loop and improve the performance of your algorithm."}, {"start": 107.48, "end": 113.36, "text": " But because decision trees, including tree ensembles, tend to be pretty fast to train,"}, {"start": 113.36, "end": 118.24000000000001, "text": " that allows you to go through this loop more quickly and maybe more efficiently improve"}, {"start": 118.24000000000001, "end": 121.0, "text": " the performance of your learning algorithm."}, {"start": 121.0, "end": 125.60000000000001, "text": " Finally, small decision trees may be human interpretable."}, {"start": 125.60000000000001, "end": 131.0, "text": " If you are training just a single decision tree and if that decision tree has only, say,"}, {"start": 131.0, "end": 136.78, "text": " a few dozen nodes, you may be able to print out the decision tree to understand exactly"}, {"start": 136.78, "end": 139.2, "text": " how it's making decisions."}, {"start": 139.2, "end": 144.72, "text": " I think that the interpretability of decision trees is sometimes a bit overstated because"}, {"start": 144.72, "end": 149.72, "text": " when you build an ensemble of a hundred trees and if each of those trees has hundreds of"}, {"start": 149.72, "end": 155.2, "text": " nodes, then looking at that ensemble to figure out what it's doing does become difficult"}, {"start": 155.2, "end": 158.24, "text": " and may need some separate visualization techniques."}, {"start": 158.24, "end": 162.92000000000002, "text": " But if you have a small decision tree, you can actually look at it and see, oh, it's"}, {"start": 162.92, "end": 169.35999999999999, "text": " classifying whether something is a cat by looking at certain features in certain ways."}, {"start": 169.35999999999999, "end": 176.35999999999999, "text": " If you've decided to use a decision tree or tree ensemble, I would probably use XGBoost"}, {"start": 176.35999999999999, "end": 179.56, "text": " for most of the applications I would work on."}, {"start": 179.56, "end": 184.51999999999998, "text": " One slight downside of a tree ensemble is that it is a bit more expensive than a single"}, {"start": 184.51999999999998, "end": 186.11999999999998, "text": " decision tree."}, {"start": 186.11999999999998, "end": 191.11999999999998, "text": " And so if you had a very, very constrained computational budget, you might use a single"}, {"start": 191.12, "end": 196.84, "text": " decision tree, but other than that setting, I would almost always use a tree ensemble"}, {"start": 196.84, "end": 198.72, "text": " and use XGBoost in particular."}, {"start": 198.72, "end": 201.12, "text": " How about neural networks?"}, {"start": 201.12, "end": 207.04, "text": " In contrast to decision trees and tree ensembles, it works well on all types of data, including"}, {"start": 207.04, "end": 212.84, "text": " tabular or structured data, as well as unstructured data, as well as mixed data that includes"}, {"start": 212.84, "end": 216.68, "text": " both structured and unstructured components."}, {"start": 216.68, "end": 222.32, "text": " Whereas on tabular structured data, neural networks and decision trees are often both"}, {"start": 222.32, "end": 228.6, "text": " competitive on unstructured data, such as images, video, audio, and text, a neural network will"}, {"start": 228.6, "end": 234.60000000000002, "text": " really be the preferred algorithm and not a decision tree or tree ensemble."}, {"start": 234.60000000000002, "end": 240.56, "text": " On the downside, though, neural networks may be slower than a decision tree."}, {"start": 240.56, "end": 245.08, "text": " A large neural network can just take a long time to train."}, {"start": 245.08, "end": 250.04000000000002, "text": " Other benefits of neural networks includes that it works with transfer learning."}, {"start": 250.04000000000002, "end": 254.68, "text": " And this is really important because for many applications where you have only a small data"}, {"start": 254.68, "end": 261.68, "text": " set, being able to use transfer learning and carry out pre-training on a much larger data"}, {"start": 261.68, "end": 266.2, "text": " set that is critical to getting competitive performance."}, {"start": 266.2, "end": 272.52000000000004, "text": " Finally, there are also some technical reasons why it might be easier to string together"}, {"start": 272.52, "end": 277.91999999999996, "text": " multiple neural networks to build a larger machine learning system."}, {"start": 277.91999999999996, "end": 285.24, "text": " And the basic reason is neural networks compute the output y as a smooth or continuous function"}, {"start": 285.24, "end": 286.47999999999996, "text": " of the input x."}, {"start": 286.47999999999996, "end": 291.52, "text": " And so even if you string together a lot of different models, the outputs of all of these"}, {"start": 291.52, "end": 294.2, "text": " different models is self-differentiable."}, {"start": 294.2, "end": 297.79999999999995, "text": " And so you can train them all at the same time using gradient descent."}, {"start": 297.79999999999995, "end": 302.08, "text": " Whereas with a decision tree, you can only train one tree at a time."}, {"start": 302.08, "end": 307.44, "text": " Finally, if you're building a system of multiple machine learning models working together,"}, {"start": 307.44, "end": 311.91999999999996, "text": " it might be easier to string together and train multiple neural networks than multiple"}, {"start": 311.91999999999996, "end": 313.76, "text": " decision trees."}, {"start": 313.76, "end": 317.28, "text": " The reasons for this are quite technical, and you don't need to worry about it for the"}, {"start": 317.28, "end": 318.28, "text": " purpose of this course."}, {"start": 318.28, "end": 323.56, "text": " But it relates to that even when you string together multiple neural networks, you can"}, {"start": 323.56, "end": 327.4, "text": " train them all together using gradient descent."}, {"start": 327.4, "end": 332.88, "text": " Whereas for decision trees, you can only train one decision tree at a time."}, {"start": 332.88, "end": 333.88, "text": " So that's it."}, {"start": 333.88, "end": 338.88, "text": " You've reached the end of the videos for this course on advanced learning algorithms."}, {"start": 338.88, "end": 342.47999999999996, "text": " Thank you for sticking with me all this way and congratulations on getting to the end"}, {"start": 342.47999999999996, "end": 345.88, "text": " of the videos on advanced learning algorithms."}, {"start": 345.88, "end": 350.67999999999995, "text": " You've now learned how to build and use both neural networks and decision trees, and also"}, {"start": 350.67999999999995, "end": 356.52, "text": " heard about a variety of tips, practical advice on how to get these algorithms to work well"}, {"start": 356.52, "end": 358.15999999999997, "text": " for you."}, {"start": 358.15999999999997, "end": 363.68, "text": " But even with all that you've seen on supervised learning, that's just part of what learning"}, {"start": 363.68, "end": 365.59999999999997, "text": " algorithms can do."}, {"start": 365.59999999999997, "end": 370.96, "text": " Supervised learnings need label datasets with the labels Y on your training set."}, {"start": 370.96, "end": 375.68, "text": " There's another set of very powerful algorithms called unsupervised learning algorithms, where"}, {"start": 375.68, "end": 381.03999999999996, "text": " you don't even need labels Y for the algorithm to figure out very interesting patterns and"}, {"start": 381.03999999999996, "end": 383.44, "text": " to do things with the data that you have."}, {"start": 383.44, "end": 389.12, "text": " So I look forward to seeing you also in the third and final course of this specialization,"}, {"start": 389.12, "end": 392.48, "text": " which will be on unsupervised learning."}, {"start": 392.48, "end": 397.76, "text": " Now before you finish up this course, I hope you also enjoy practicing the ideas of decision"}, {"start": 397.76, "end": 401.68, "text": " trees in their practice quizzes and in their practice labs."}, {"start": 401.68, "end": 405.4, "text": " I'd like to wish you the best of luck in the practice labs."}, {"start": 405.4, "end": 413.79999999999995, "text": " For those of you that may be Star Wars fans, let me say may the forest be with you."}]