2.10 Practical Tips for Linear Regression | Polynomial regression --[Machine Learning | Andrew Ng]
 So far, we've just been fitting straight lines to our data. Let's take the ideas of multiple linear regression and feature engineering to come up with a new algorithm called polynomial regression, which will let you fit curves, nonlinear functions, to your data. Let's say you have a housing data set that looks like this, where feature x is the size in square feet. It doesn't look like a straight line fits this data set very well, so maybe you want to fit a curve, maybe a quadratic function, to the data like this, which includes a size x and also x squared, which is the size raised to the power of 2. And maybe that will give you a better fit to the data. But then you may decide that your quadratic model doesn't really make sense, because a quadratic function eventually comes back down, and well, we wouldn't really expect housing prices to go down when the size increases, right? Big houses seem like they should usually cost more. So then you may choose a cubic function, where we now have not only x squared, but x cubed. So maybe this model produces this curve here, which is a somewhat better fit to the data, because the size does eventually come back up as the size increases. These are both examples of polynomial regression, because you took your optional feature x and raised it to the power of 2, or 3, or any other power. And in the case of the cubic function, the first feature is the size, the second feature is the size squared, and the third feature is the size cubed. I just want to point out one more thing, which is that if you create features that are these powers, like the square of the original features like this, then feature scaling becomes increasingly important. So if the size of the house ranges from say, 1 to 1000 square feet, then the second feature, which is the size squared, would range from 1 to a million, and the third feature, which is size cubed, ranges from 1 to a billion. So these two features, x squared and x cubed, take on very different ranges of values compared to the original feature x. And if you're using gradient descent, it's important to apply feature scaling to get your features into comparable ranges of values. Finally, here's one last example of how you really have a wide range of choices of features to use. Another reasonable alternative to taking the size squared and size cubed is to say use the square root of x. So your model may look like w1 times x plus w2 times the square root of x plus b. The square root function looks like this, and it becomes a bit less steep as x increases, but it doesn't ever completely flatten out, and it certainly never ever comes back down. So this would be another choice of features that might work well for this dataset as well. So you may ask yourself, how do I decide what features to use? Later, in the second course in this specialization, you see how you can choose different features and different models that include or don't include these features, and you have a process for measuring how well these different models perform to help you decide which features to include or not include. For now, I just want you to be aware that you have a choice in what features you use, and by using feature engineering and polynomial functions, you can potentially get a much better model for your data. In the optional lab that follows this video, you will see some code that implements polynomial regression using features like x, x squared, and x cubed. So please take a look and run the code and see how it works. There's also another optional lab after that one that shows how to use a popular open source toolkit that implements linear regression. Scikit-learn is a very widely used open source machine learning library that is used by many practitioners in many of the top AI, internet, machine learning companies in the world. So if either now or in the future you're using machine learning in your job, there's a very good chance you'll be using tools like Scikit-learn to train your models. And so working through that optional lab will give you a chance to not only better understand linear regression, but also see how this can be done in just a few lines of code using a library like Scikit-learn. For you to have a solid understanding of these algorithms and be able to apply them, I do think it's important that you know how to implement linear regression yourself, and not just call some Scikit-learn function that is a black box. But Scikit-learn also has an important role in the way machine learning is done in practice today. So, we're just about at the end of this week. Congratulations on finishing all of this week's videos. Please do take a look at the practice quizzes and also the practice lab, which I hope will let you try out and practice ideas that we've discussed. In this week's practice lab, you implement linear regression. I hope you have a lot of fun getting this learning algorithm to work for yourself. Best of luck with that, and I also look forward to seeing you in next week's videos, where we'll go beyond regression, that is predicting numbers, to talk about our first classification algorithm, which can predict categories. I'll see you next week.
[{"start": 0.0, "end": 7.54, "text": " So far, we've just been fitting straight lines to our data."}, {"start": 7.54, "end": 12.56, "text": " Let's take the ideas of multiple linear regression and feature engineering to come up with a"}, {"start": 12.56, "end": 18.04, "text": " new algorithm called polynomial regression, which will let you fit curves, nonlinear functions,"}, {"start": 18.04, "end": 19.04, "text": " to your data."}, {"start": 19.04, "end": 24.6, "text": " Let's say you have a housing data set that looks like this, where feature x is the size"}, {"start": 24.6, "end": 26.400000000000002, "text": " in square feet."}, {"start": 26.4, "end": 31.2, "text": " It doesn't look like a straight line fits this data set very well, so maybe you want"}, {"start": 31.2, "end": 38.6, "text": " to fit a curve, maybe a quadratic function, to the data like this, which includes a size"}, {"start": 38.6, "end": 44.56, "text": " x and also x squared, which is the size raised to the power of 2."}, {"start": 44.56, "end": 48.239999999999995, "text": " And maybe that will give you a better fit to the data."}, {"start": 48.239999999999995, "end": 51.84, "text": " But then you may decide that your quadratic model doesn't really make sense, because a"}, {"start": 51.84, "end": 57.400000000000006, "text": " quadratic function eventually comes back down, and well, we wouldn't really expect housing"}, {"start": 57.400000000000006, "end": 61.0, "text": " prices to go down when the size increases, right?"}, {"start": 61.0, "end": 64.88000000000001, "text": " Big houses seem like they should usually cost more."}, {"start": 64.88000000000001, "end": 72.66, "text": " So then you may choose a cubic function, where we now have not only x squared, but x cubed."}, {"start": 72.66, "end": 78.66, "text": " So maybe this model produces this curve here, which is a somewhat better fit to the data,"}, {"start": 78.66, "end": 83.8, "text": " because the size does eventually come back up as the size increases."}, {"start": 83.8, "end": 89.75999999999999, "text": " These are both examples of polynomial regression, because you took your optional feature x and"}, {"start": 89.75999999999999, "end": 94.84, "text": " raised it to the power of 2, or 3, or any other power."}, {"start": 94.84, "end": 99.5, "text": " And in the case of the cubic function, the first feature is the size, the second feature"}, {"start": 99.5, "end": 104.44, "text": " is the size squared, and the third feature is the size cubed."}, {"start": 104.44, "end": 109.48, "text": " I just want to point out one more thing, which is that if you create features that are these"}, {"start": 109.48, "end": 116.32, "text": " powers, like the square of the original features like this, then feature scaling becomes increasingly"}, {"start": 116.32, "end": 117.67999999999999, "text": " important."}, {"start": 117.67999999999999, "end": 124.72, "text": " So if the size of the house ranges from say, 1 to 1000 square feet, then the second feature,"}, {"start": 124.72, "end": 130.72, "text": " which is the size squared, would range from 1 to a million, and the third feature, which"}, {"start": 130.72, "end": 135.24, "text": " is size cubed, ranges from 1 to a billion."}, {"start": 135.24, "end": 141.44, "text": " So these two features, x squared and x cubed, take on very different ranges of values compared"}, {"start": 141.44, "end": 143.28, "text": " to the original feature x."}, {"start": 143.28, "end": 148.07999999999998, "text": " And if you're using gradient descent, it's important to apply feature scaling to get"}, {"start": 148.07999999999998, "end": 152.0, "text": " your features into comparable ranges of values."}, {"start": 152.0, "end": 157.96, "text": " Finally, here's one last example of how you really have a wide range of choices of features"}, {"start": 157.96, "end": 159.64, "text": " to use."}, {"start": 159.64, "end": 164.79999999999998, "text": " Another reasonable alternative to taking the size squared and size cubed is to say use"}, {"start": 164.79999999999998, "end": 166.79999999999998, "text": " the square root of x."}, {"start": 166.79999999999998, "end": 175.88, "text": " So your model may look like w1 times x plus w2 times the square root of x plus b."}, {"start": 175.88, "end": 182.07999999999998, "text": " The square root function looks like this, and it becomes a bit less steep as x increases,"}, {"start": 182.07999999999998, "end": 187.85999999999999, "text": " but it doesn't ever completely flatten out, and it certainly never ever comes back down."}, {"start": 187.86, "end": 193.20000000000002, "text": " So this would be another choice of features that might work well for this dataset as well."}, {"start": 193.20000000000002, "end": 197.76000000000002, "text": " So you may ask yourself, how do I decide what features to use?"}, {"start": 197.76000000000002, "end": 203.5, "text": " Later, in the second course in this specialization, you see how you can choose different features"}, {"start": 203.5, "end": 209.0, "text": " and different models that include or don't include these features, and you have a process"}, {"start": 209.0, "end": 213.52, "text": " for measuring how well these different models perform to help you decide which features"}, {"start": 213.52, "end": 215.88000000000002, "text": " to include or not include."}, {"start": 215.88, "end": 221.07999999999998, "text": " For now, I just want you to be aware that you have a choice in what features you use,"}, {"start": 221.07999999999998, "end": 225.79999999999998, "text": " and by using feature engineering and polynomial functions, you can potentially get a much"}, {"start": 225.79999999999998, "end": 229.0, "text": " better model for your data."}, {"start": 229.0, "end": 234.48, "text": " In the optional lab that follows this video, you will see some code that implements polynomial"}, {"start": 234.48, "end": 238.72, "text": " regression using features like x, x squared, and x cubed."}, {"start": 238.72, "end": 243.84, "text": " So please take a look and run the code and see how it works."}, {"start": 243.84, "end": 248.72, "text": " There's also another optional lab after that one that shows how to use a popular open source"}, {"start": 248.72, "end": 253.92000000000002, "text": " toolkit that implements linear regression."}, {"start": 253.92000000000002, "end": 259.28000000000003, "text": " Scikit-learn is a very widely used open source machine learning library that is used by many"}, {"start": 259.28000000000003, "end": 266.92, "text": " practitioners in many of the top AI, internet, machine learning companies in the world."}, {"start": 266.92, "end": 271.68, "text": " So if either now or in the future you're using machine learning in your job, there's a very"}, {"start": 271.68, "end": 277.76, "text": " good chance you'll be using tools like Scikit-learn to train your models."}, {"start": 277.76, "end": 282.6, "text": " And so working through that optional lab will give you a chance to not only better understand"}, {"start": 282.6, "end": 288.0, "text": " linear regression, but also see how this can be done in just a few lines of code using"}, {"start": 288.0, "end": 291.32, "text": " a library like Scikit-learn."}, {"start": 291.32, "end": 296.52, "text": " For you to have a solid understanding of these algorithms and be able to apply them, I do"}, {"start": 296.52, "end": 301.12, "text": " think it's important that you know how to implement linear regression yourself, and"}, {"start": 301.12, "end": 305.12, "text": " not just call some Scikit-learn function that is a black box."}, {"start": 305.12, "end": 309.92, "text": " But Scikit-learn also has an important role in the way machine learning is done in practice"}, {"start": 309.92, "end": 310.92, "text": " today."}, {"start": 310.92, "end": 315.04, "text": " So, we're just about at the end of this week."}, {"start": 315.04, "end": 318.4, "text": " Congratulations on finishing all of this week's videos."}, {"start": 318.4, "end": 323.24, "text": " Please do take a look at the practice quizzes and also the practice lab, which I hope will"}, {"start": 323.24, "end": 327.68, "text": " let you try out and practice ideas that we've discussed."}, {"start": 327.68, "end": 331.54, "text": " In this week's practice lab, you implement linear regression."}, {"start": 331.54, "end": 336.6, "text": " I hope you have a lot of fun getting this learning algorithm to work for yourself."}, {"start": 336.6, "end": 342.0, "text": " Best of luck with that, and I also look forward to seeing you in next week's videos, where"}, {"start": 342.0, "end": 346.96000000000004, "text": " we'll go beyond regression, that is predicting numbers, to talk about our first classification"}, {"start": 346.96000000000004, "end": 349.72, "text": " algorithm, which can predict categories."}, {"start": 349.72, "end": 358.12, "text": " I'll see you next week."}]