7.6 Decision Tree Learning | Using one-hot encoding of categorical features --[ML | Andrew Ng]
 In the example we've seen so far, each of the features could take on only one of two possible values. The ear shape was either pointy or floppy, the face shape was either round or not round, and whiskers were either present or absent. But what if you have features that can take on more than two discrete values? In this video, we'll look at how you can use one-hot encoding to adjust features like that. Here's a new training set for our Pet Adoption Center application, where all the data is the same except for the ear shape feature. Rather than the ear shape only being pointy and floppy, it can now also take on an oval shape. And so the ear shape feature is still a categorical value feature, but it can take on three possible values instead of just two possible values. And this means that when you split on this feature, you end up creating three subsets of the data and end up building three sub-branches for this tree. But in this video, I'd like to describe a different way of adjusting features that can take on more than two values, which is to use a one-hot encoding. In particular, rather than using an ear shape feature that can take on any of three possible values, we're instead going to create three new features where one feature is, does this animal have pointy ears? A second is, does it have floppy ears? And the third is, does it have oval ears? And so for the first example, whereas we previously had ear shape as pointy, we will now instead say that this animal has a value for the pointy ear feature of one and zero for floppy and oval. Whereas previously for the second example, we previously said it had oval ears. Now we'll say that it has a value of zero for pointy ears because it doesn't have pointy ears. It also doesn't have floppy ears, but it does have oval ears, which is why this value here is one and so on for the rest of the examples in the dataset. And so instead of one feature taking on three possible values, we've now constructed three new features, each of which can take on only one of two possible values, either zero or one. In a little bit more detail, if a categorical feature can take on k possible values, k was three in our example, then we will replace it by creating k binary features that can only take on the values zero or one. And you notice that among all of these three features, if you look at any row here, exactly one of the values is equal to one. And that's what gives this method of feature construction the name one hot encoding. And because one of these features will always take on the value one, that's the hot feature and hence the name one hot encoding. And with this choice of features, we're now back to the original setting of where each feature only takes on one of two possible values. And so the decision tree learning algorithm that we've seen previously will apply to this data with no further modifications. Just as an aside, even though this week's material has been focused on training decision tree models, the idea of using one hot encodings to encode categorical features also works for training neural networks. In particular, if you were to take the face shape feature and replace round and not round with one and zero, where round gets mapped to one, not round gets mapped to zero, and so on. And for whiskers, similarly, replace presence with one and absence with zero. Then notice that we have taken all the categorical features we had where we had three possible values for ear shape, two for face shape and one for whiskers and encoded as a list of these five features, three from the one hot encoding of ear shape, one from face shape and one from whiskers. And now this list of five features can also be fed to a neural network or to logistic regression to try to train a cat classifier. So one hot encoding is a technique that works not just for decision tree learning, but also lets you encode categorical features using ones and zeros so that it can be fed as inputs to a neural network as well, which expects numbers as inputs. So that's it, with a one hot encoding you can get your decision tree to work on features that can take on more than two discrete values. And you can also apply this to neural network or linear regression or logistic regression training. But how about features that are numbers that can take on any value, not just a small number of discrete values? In the next video, let's look at how you can get the decision tree to handle continuous value features that can be any number.
[{"start": 0.0, "end": 7.5200000000000005, "text": " In the example we've seen so far, each of the features could take on only one of two"}, {"start": 7.5200000000000005, "end": 9.32, "text": " possible values."}, {"start": 9.32, "end": 14.32, "text": " The ear shape was either pointy or floppy, the face shape was either round or not round,"}, {"start": 14.32, "end": 17.22, "text": " and whiskers were either present or absent."}, {"start": 17.22, "end": 22.64, "text": " But what if you have features that can take on more than two discrete values?"}, {"start": 22.64, "end": 28.0, "text": " In this video, we'll look at how you can use one-hot encoding to adjust features like that."}, {"start": 28.0, "end": 34.4, "text": " Here's a new training set for our Pet Adoption Center application, where all the data is"}, {"start": 34.4, "end": 37.76, "text": " the same except for the ear shape feature."}, {"start": 37.76, "end": 45.0, "text": " Rather than the ear shape only being pointy and floppy, it can now also take on an oval"}, {"start": 45.0, "end": 46.0, "text": " shape."}, {"start": 46.0, "end": 53.72, "text": " And so the ear shape feature is still a categorical value feature, but it can take on three possible"}, {"start": 53.72, "end": 57.519999999999996, "text": " values instead of just two possible values."}, {"start": 57.52, "end": 62.480000000000004, "text": " And this means that when you split on this feature, you end up creating three subsets"}, {"start": 62.480000000000004, "end": 70.36, "text": " of the data and end up building three sub-branches for this tree."}, {"start": 70.36, "end": 76.36, "text": " But in this video, I'd like to describe a different way of adjusting features that can"}, {"start": 76.36, "end": 82.10000000000001, "text": " take on more than two values, which is to use a one-hot encoding."}, {"start": 82.1, "end": 87.83999999999999, "text": " In particular, rather than using an ear shape feature that can take on any of three possible"}, {"start": 87.83999999999999, "end": 96.28, "text": " values, we're instead going to create three new features where one feature is, does this"}, {"start": 96.28, "end": 98.08, "text": " animal have pointy ears?"}, {"start": 98.08, "end": 100.39999999999999, "text": " A second is, does it have floppy ears?"}, {"start": 100.39999999999999, "end": 103.72, "text": " And the third is, does it have oval ears?"}, {"start": 103.72, "end": 111.02, "text": " And so for the first example, whereas we previously had ear shape as pointy, we will now instead"}, {"start": 111.02, "end": 118.72, "text": " say that this animal has a value for the pointy ear feature of one and zero for floppy and"}, {"start": 118.72, "end": 120.03999999999999, "text": " oval."}, {"start": 120.03999999999999, "end": 125.5, "text": " Whereas previously for the second example, we previously said it had oval ears."}, {"start": 125.5, "end": 131.2, "text": " Now we'll say that it has a value of zero for pointy ears because it doesn't have pointy"}, {"start": 131.2, "end": 132.2, "text": " ears."}, {"start": 132.2, "end": 136.88, "text": " It also doesn't have floppy ears, but it does have oval ears, which is why this value here"}, {"start": 136.88, "end": 141.68, "text": " is one and so on for the rest of the examples in the dataset."}, {"start": 141.68, "end": 147.44, "text": " And so instead of one feature taking on three possible values, we've now constructed three"}, {"start": 147.44, "end": 153.4, "text": " new features, each of which can take on only one of two possible values, either zero or"}, {"start": 153.4, "end": 154.4, "text": " one."}, {"start": 154.4, "end": 162.2, "text": " In a little bit more detail, if a categorical feature can take on k possible values, k was"}, {"start": 162.2, "end": 169.16, "text": " three in our example, then we will replace it by creating k binary features that can"}, {"start": 169.16, "end": 172.64, "text": " only take on the values zero or one."}, {"start": 172.64, "end": 179.23999999999998, "text": " And you notice that among all of these three features, if you look at any row here, exactly"}, {"start": 179.23999999999998, "end": 183.07999999999998, "text": " one of the values is equal to one."}, {"start": 183.07999999999998, "end": 188.67999999999998, "text": " And that's what gives this method of feature construction the name one hot encoding."}, {"start": 188.68, "end": 194.48000000000002, "text": " And because one of these features will always take on the value one, that's the hot feature"}, {"start": 194.48000000000002, "end": 197.44, "text": " and hence the name one hot encoding."}, {"start": 197.44, "end": 202.12, "text": " And with this choice of features, we're now back to the original setting of where each"}, {"start": 202.12, "end": 205.54000000000002, "text": " feature only takes on one of two possible values."}, {"start": 205.54000000000002, "end": 209.64000000000001, "text": " And so the decision tree learning algorithm that we've seen previously will apply to this"}, {"start": 209.64000000000001, "end": 213.52, "text": " data with no further modifications."}, {"start": 213.52, "end": 219.0, "text": " Just as an aside, even though this week's material has been focused on training decision"}, {"start": 219.0, "end": 225.92000000000002, "text": " tree models, the idea of using one hot encodings to encode categorical features also works"}, {"start": 225.92000000000002, "end": 228.84, "text": " for training neural networks."}, {"start": 228.84, "end": 234.16000000000003, "text": " In particular, if you were to take the face shape feature and replace round and not round"}, {"start": 234.16000000000003, "end": 241.16000000000003, "text": " with one and zero, where round gets mapped to one, not round gets mapped to zero, and"}, {"start": 241.16000000000003, "end": 242.16000000000003, "text": " so on."}, {"start": 242.16, "end": 248.88, "text": " And for whiskers, similarly, replace presence with one and absence with zero."}, {"start": 248.88, "end": 254.56, "text": " Then notice that we have taken all the categorical features we had where we had three possible"}, {"start": 254.56, "end": 260.64, "text": " values for ear shape, two for face shape and one for whiskers and encoded as a list of"}, {"start": 260.64, "end": 267.04, "text": " these five features, three from the one hot encoding of ear shape, one from face shape"}, {"start": 267.04, "end": 268.78, "text": " and one from whiskers."}, {"start": 268.78, "end": 273.96, "text": " And now this list of five features can also be fed to a neural network or to logistic"}, {"start": 273.96, "end": 277.52, "text": " regression to try to train a cat classifier."}, {"start": 277.52, "end": 283.5, "text": " So one hot encoding is a technique that works not just for decision tree learning, but also"}, {"start": 283.5, "end": 290.47999999999996, "text": " lets you encode categorical features using ones and zeros so that it can be fed as inputs"}, {"start": 290.47999999999996, "end": 295.08, "text": " to a neural network as well, which expects numbers as inputs."}, {"start": 295.08, "end": 299.88, "text": " So that's it, with a one hot encoding you can get your decision tree to work on features"}, {"start": 299.88, "end": 303.47999999999996, "text": " that can take on more than two discrete values."}, {"start": 303.47999999999996, "end": 308.44, "text": " And you can also apply this to neural network or linear regression or logistic regression"}, {"start": 308.44, "end": 310.2, "text": " training."}, {"start": 310.2, "end": 316.15999999999997, "text": " But how about features that are numbers that can take on any value, not just a small number"}, {"start": 316.15999999999997, "end": 317.84, "text": " of discrete values?"}, {"start": 317.84, "end": 322.52, "text": " In the next video, let's look at how you can get the decision tree to handle continuous"}, {"start": 322.52, "end": 325.44, "text": " value features that can be any number."}]