2.6 Practical Tips for Linear Regression | Feature scaling part 2-- [Machine Learning | Andrew Ng]
 Let's look at how you can implement feature scaling to take features that take on very different ranges of values and scale them to have comparable ranges of value to each other. So how do you actually scale features? Well, if X1 ranges from 3 to 2000, one way to get a scale version of X1 is to take each original X1 value and divide by 2000 the maximum of the range. So the scale X1 will range from 0.15 up to 1. Similarly, since X2 ranges from 0 to 5, you can calculate a scale version of X2 by taking each original X2 and dividing by 5, which is again the maximum. So the scale X2 will now range from 0 to 1. So if you plot the scaled X1 and X2 on a graph, it might look like this. In addition to dividing by the maximum, you can also do what's called mean normalization. So what this looks like is, you start with the original features and then you rescale them so that both of them are centered around 0. So whereas before they only had values greater than 0, now they have both negative and positive values, but maybe usually between negative 1 and plus 1. So to calculate the mean normalization of X1, first find the average, also called the mean of X1 on your training set, and let's call this mean mu1, with this being the Greek alphabet mu. For example, you may find that the average of feature 1, mu1, is 600 square feet. So let's take each X1, subtract the mean mu1, and then let's divide by the difference 2000 minus 300, where 2000 is the maximum and 300 the minimum. And if you do this, you get the normalized X1 to range from negative 0.18 to 0.82. Similarly to mean normalize X2, you can calculate the average of feature 2, and for instance mu2 may be 2.3. Then you can take each X2, subtract mu2, and divide by 5 minus 0, again the max 5 minus the min which is 0. The mean normalized X2 now ranges from negative 0.46 to 0.54, so if you plot the training data using the mean normalized X1 and X2, it might look like this. There's one last common rescaling method called z-score normalization. To implement z-score normalization, you need to calculate something called the standard deviation of each feature. If you don't know what a standard deviation is, don't worry about it, you won't need to know it for this class. Or if you've heard of the normal distribution or the bell-shaped curve, sometimes also called the Gaussian distribution, this is what the standard deviation for the normal distribution looks like. But if you haven't heard of this, you don't need to worry about that either. But if you do know what is the standard deviation, then to implement a z-score normalization, you first calculate the mean mu as well as the standard deviation, which is often denoted by the lowercase Greek alphabet sigma of each feature. So for instance, maybe feature 1 has a standard deviation of 450 and mean 600, then to z-score normalize x1, take each x1, subtract mu 1, and then divide by the standard deviation, which I'm going to denote as sigma 1. And what you might find is that the z-score normalize x1 now ranges from negative 0.67 to 3.1. Similarly, if you calculate the second feature's standard deviation to be 1.4 and mean to be 2.3, then you can compute x2 minus mu 2 divided by sigma 2. And in this case, the z-score normalize by x2 might now range from negative 1.6 to 1.9. So if you plot the training data on the normalized x1 and x2 on a graph, it might look like this. As a rule of thumb, when performing feature scaling, you might want to aim for getting the features to range from maybe anywhere around negative 1 to somewhere around plus 1 for each feature x. But these values, negative 1 and plus 1 can be a little bit loose. So if the features range from negative 3 to plus 3, or negative 0.3 to plus 0.3, all of these are completely okay. So if you have a feature x1 that winds up being between 0 and 3, that's not a problem. And you can rescale it if you want, but if you don't rescale it, it should work okay too. Or if you have a different feature, x2, whose values are between negative 2 and plus 0.5, again, that's okay. No harm rescaling it, but it might be okay if you leave it alone as well. But if another feature like x3 here ranges from negative 100 to plus 100, then this takes on a very different range of values than something from around negative 1 to plus 1. So you're probably better off rescaling this feature x3 so that it ranges from something closer to negative 1 to plus 1. Similarly, if you have a feature x4 that takes on really small values, say between negative 0.001 and plus 0.001, then these values are so small, that means you may want to rescale it as well. Finally, what if your feature x5, such as measurements of a hospital patient's body temperature, ranges from 98.6 to 105 degrees Fahrenheit? In this case, these values are around 100, which is actually pretty large compared to other scale features, and this will actually cause gradient descent to run more slowly. So in this case, feature rescaling will likely help. There's almost never any harm to carrying out feature rescaling, so when in doubt, I encourage you to just carry it out. And that's it for feature scaling. With this little technique, you'll often be able to get gradient descent to run much faster. So that's feature scaling. And with or without feature scaling, when you run gradient descent, how can you know, how can you check if gradient descent is really working, if it is finding you the global minimum or something close to it? In the next video, let's take a look at how to recognize if gradient descent is converging. And then in the video after that, this will lead to discussion of how to choose a good learning rate for gradient descent.
[{"start": 0.0, "end": 6.6000000000000005, "text": " Let's look at how you can implement feature scaling to take features that take on very"}, {"start": 6.6000000000000005, "end": 11.28, "text": " different ranges of values and scale them to have comparable ranges of value to each"}, {"start": 11.28, "end": 12.280000000000001, "text": " other."}, {"start": 12.280000000000001, "end": 14.56, "text": " So how do you actually scale features?"}, {"start": 14.56, "end": 22.76, "text": " Well, if X1 ranges from 3 to 2000, one way to get a scale version of X1 is to take each"}, {"start": 22.76, "end": 28.92, "text": " original X1 value and divide by 2000 the maximum of the range."}, {"start": 28.92, "end": 34.440000000000005, "text": " So the scale X1 will range from 0.15 up to 1."}, {"start": 34.440000000000005, "end": 41.760000000000005, "text": " Similarly, since X2 ranges from 0 to 5, you can calculate a scale version of X2 by taking"}, {"start": 41.760000000000005, "end": 47.2, "text": " each original X2 and dividing by 5, which is again the maximum."}, {"start": 47.2, "end": 52.56, "text": " So the scale X2 will now range from 0 to 1."}, {"start": 52.56, "end": 59.0, "text": " So if you plot the scaled X1 and X2 on a graph, it might look like this."}, {"start": 59.0, "end": 65.12, "text": " In addition to dividing by the maximum, you can also do what's called mean normalization."}, {"start": 65.12, "end": 69.72, "text": " So what this looks like is, you start with the original features and then you rescale"}, {"start": 69.72, "end": 73.52000000000001, "text": " them so that both of them are centered around 0."}, {"start": 73.52000000000001, "end": 79.24000000000001, "text": " So whereas before they only had values greater than 0, now they have both negative and positive"}, {"start": 79.24, "end": 85.67999999999999, "text": " values, but maybe usually between negative 1 and plus 1."}, {"start": 85.67999999999999, "end": 91.0, "text": " So to calculate the mean normalization of X1, first find the average, also called the"}, {"start": 91.0, "end": 98.0, "text": " mean of X1 on your training set, and let's call this mean mu1, with this being the Greek"}, {"start": 98.0, "end": 99.8, "text": " alphabet mu."}, {"start": 99.8, "end": 106.74, "text": " For example, you may find that the average of feature 1, mu1, is 600 square feet."}, {"start": 106.74, "end": 115.24, "text": " So let's take each X1, subtract the mean mu1, and then let's divide by the difference 2000"}, {"start": 115.24, "end": 122.08, "text": " minus 300, where 2000 is the maximum and 300 the minimum."}, {"start": 122.08, "end": 131.4, "text": " And if you do this, you get the normalized X1 to range from negative 0.18 to 0.82."}, {"start": 131.4, "end": 137.96, "text": " Similarly to mean normalize X2, you can calculate the average of feature 2, and for instance"}, {"start": 137.96, "end": 141.12, "text": " mu2 may be 2.3."}, {"start": 141.12, "end": 150.4, "text": " Then you can take each X2, subtract mu2, and divide by 5 minus 0, again the max 5 minus"}, {"start": 150.4, "end": 153.0, "text": " the min which is 0."}, {"start": 153.0, "end": 162.76, "text": " The mean normalized X2 now ranges from negative 0.46 to 0.54, so if you plot the training"}, {"start": 162.76, "end": 169.08, "text": " data using the mean normalized X1 and X2, it might look like this."}, {"start": 169.08, "end": 174.52, "text": " There's one last common rescaling method called z-score normalization."}, {"start": 174.52, "end": 178.8, "text": " To implement z-score normalization, you need to calculate something called the standard"}, {"start": 178.8, "end": 181.04, "text": " deviation of each feature."}, {"start": 181.04, "end": 184.6, "text": " If you don't know what a standard deviation is, don't worry about it, you won't need to"}, {"start": 184.6, "end": 186.64, "text": " know it for this class."}, {"start": 186.64, "end": 191.12, "text": " Or if you've heard of the normal distribution or the bell-shaped curve, sometimes also called"}, {"start": 191.12, "end": 196.72, "text": " the Gaussian distribution, this is what the standard deviation for the normal distribution"}, {"start": 196.72, "end": 197.72, "text": " looks like."}, {"start": 197.72, "end": 201.28, "text": " But if you haven't heard of this, you don't need to worry about that either."}, {"start": 201.28, "end": 207.04, "text": " But if you do know what is the standard deviation, then to implement a z-score normalization,"}, {"start": 207.04, "end": 213.48, "text": " you first calculate the mean mu as well as the standard deviation, which is often denoted"}, {"start": 213.48, "end": 218.68, "text": " by the lowercase Greek alphabet sigma of each feature."}, {"start": 218.68, "end": 228.35999999999999, "text": " So for instance, maybe feature 1 has a standard deviation of 450 and mean 600, then to z-score"}, {"start": 228.36, "end": 236.96, "text": " normalize x1, take each x1, subtract mu 1, and then divide by the standard deviation,"}, {"start": 236.96, "end": 240.20000000000002, "text": " which I'm going to denote as sigma 1."}, {"start": 240.20000000000002, "end": 247.92000000000002, "text": " And what you might find is that the z-score normalize x1 now ranges from negative 0.67"}, {"start": 247.92000000000002, "end": 250.92000000000002, "text": " to 3.1."}, {"start": 250.92, "end": 258.52, "text": " Similarly, if you calculate the second feature's standard deviation to be 1.4 and mean to be"}, {"start": 258.52, "end": 266.15999999999997, "text": " 2.3, then you can compute x2 minus mu 2 divided by sigma 2."}, {"start": 266.15999999999997, "end": 275.88, "text": " And in this case, the z-score normalize by x2 might now range from negative 1.6 to 1.9."}, {"start": 275.88, "end": 283.84, "text": " So if you plot the training data on the normalized x1 and x2 on a graph, it might look like this."}, {"start": 283.84, "end": 288.88, "text": " As a rule of thumb, when performing feature scaling, you might want to aim for getting"}, {"start": 288.88, "end": 295.0, "text": " the features to range from maybe anywhere around negative 1 to somewhere around plus"}, {"start": 295.0, "end": 298.24, "text": " 1 for each feature x."}, {"start": 298.24, "end": 303.28, "text": " But these values, negative 1 and plus 1 can be a little bit loose."}, {"start": 303.28, "end": 311.11999999999995, "text": " So if the features range from negative 3 to plus 3, or negative 0.3 to plus 0.3, all of"}, {"start": 311.11999999999995, "end": 313.03999999999996, "text": " these are completely okay."}, {"start": 313.03999999999996, "end": 319.28, "text": " So if you have a feature x1 that winds up being between 0 and 3, that's not a problem."}, {"start": 319.28, "end": 323.71999999999997, "text": " And you can rescale it if you want, but if you don't rescale it, it should work okay"}, {"start": 323.71999999999997, "end": 325.14, "text": " too."}, {"start": 325.14, "end": 331.52, "text": " Or if you have a different feature, x2, whose values are between negative 2 and plus 0.5,"}, {"start": 331.52, "end": 333.4, "text": " again, that's okay."}, {"start": 333.4, "end": 339.28, "text": " No harm rescaling it, but it might be okay if you leave it alone as well."}, {"start": 339.28, "end": 346.79999999999995, "text": " But if another feature like x3 here ranges from negative 100 to plus 100, then this takes"}, {"start": 346.79999999999995, "end": 352.24, "text": " on a very different range of values than something from around negative 1 to plus 1."}, {"start": 352.24, "end": 357.82, "text": " So you're probably better off rescaling this feature x3 so that it ranges from something"}, {"start": 357.82, "end": 361.71999999999997, "text": " closer to negative 1 to plus 1."}, {"start": 361.71999999999997, "end": 368.4, "text": " Similarly, if you have a feature x4 that takes on really small values, say between negative"}, {"start": 368.4, "end": 376.84, "text": " 0.001 and plus 0.001, then these values are so small, that means you may want to rescale"}, {"start": 376.84, "end": 378.48, "text": " it as well."}, {"start": 378.48, "end": 385.4, "text": " Finally, what if your feature x5, such as measurements of a hospital patient's body"}, {"start": 385.4, "end": 392.52, "text": " temperature, ranges from 98.6 to 105 degrees Fahrenheit?"}, {"start": 392.52, "end": 398.59999999999997, "text": " In this case, these values are around 100, which is actually pretty large compared to"}, {"start": 398.59999999999997, "end": 404.44, "text": " other scale features, and this will actually cause gradient descent to run more slowly."}, {"start": 404.44, "end": 408.67999999999995, "text": " So in this case, feature rescaling will likely help."}, {"start": 408.67999999999995, "end": 413.88, "text": " There's almost never any harm to carrying out feature rescaling, so when in doubt, I"}, {"start": 413.88, "end": 416.71999999999997, "text": " encourage you to just carry it out."}, {"start": 416.71999999999997, "end": 418.96, "text": " And that's it for feature scaling."}, {"start": 418.96, "end": 425.46, "text": " With this little technique, you'll often be able to get gradient descent to run much faster."}, {"start": 425.46, "end": 427.88, "text": " So that's feature scaling."}, {"start": 427.88, "end": 432.76, "text": " And with or without feature scaling, when you run gradient descent, how can you know,"}, {"start": 432.76, "end": 438.48, "text": " how can you check if gradient descent is really working, if it is finding you the global minimum"}, {"start": 438.48, "end": 440.4, "text": " or something close to it?"}, {"start": 440.4, "end": 446.67999999999995, "text": " In the next video, let's take a look at how to recognize if gradient descent is converging."}, {"start": 446.67999999999995, "end": 451.15999999999997, "text": " And then in the video after that, this will lead to discussion of how to choose a good"}, {"start": 451.16, "end": 471.16, "text": " learning rate for gradient descent."}]