1.20 Machine Learning Overview | Running gradient descent  --[Machine Learning | Andrew Ng]
 Let's see what happens when you run gradient descent for linear regression. Let's go see the algorithm in action. Here's a plot of the model and data on the upper left, and a contour plot of the cost function on the upper right. And at the bottom is the surface plot of the same cost function. Often W and B will both be initialized to zero, but for this demonstration, let's initialize W to be equal to negative 0.1 and B to be 900. So this corresponds to f of x equals negative 0.1 x plus 900. Now if we take one step using gradient descent, we end up going from this point of the cost function out here to this point just down and to the right. And notice that the straight line fit has also changed a bit. Let's take another step. The cost function has now moved to this third point, and again the function f of x has also changed a bit. As you take more of these steps, the cost is decreasing at each update, so the parameters W and B are following this trajectory. And if you look on the left, you get this corresponding straight line fit that, you know, fits the data better and better until we've reached the global minimum. The global minimum corresponds to this straight line fit, which is a relatively good fit to the data. I mean, isn't that cool? And so that's gradient descent, and we're going to use this to fit a model to the housing data. And you can now use this f of x model to predict the price of your client's house or anyone else's house. For instance, if your friend's house size is 1250 square feet, you can now read off the value and predict that maybe they could get, I don't know, $250,000 for the house. To be more precise, this gradient descent process is called batch gradient descent. The term batch gradient descent refers to the fact that on every step of gradient descent, we're looking at all of the training examples instead of just a subset of the training data. So in computing gradient descent, when computing derivatives, we're computing the sum from i equals one to m, and batch gradient descent is looking at the entire batch of training examples at each update. I know that batch gradient descent may not be the most intuitive name, but this is what people in the machine learning community call it. If you've heard of the newsletter, The Batch, that's published by deeplearning.ai, the newsletter, The Batch, was also named for this concept in machine learning. And then it turns out that there are other versions of gradient descent that do not look at the entire training set, but instead looks at smaller subsets of the training data at each update step. But we'll use batch gradient descent for linear regression. So that's it for linear regression. Congratulations on getting through your first machine learning model. I hope you go and celebrate or, I don't know, maybe take a nap in your hammock. In the optional lab that follows this video, you see a review of the gradient descent algorithm as well as how to implement it in code. You also see a plot that shows how the cost decreases as you continue training more iterations. And you also see a contour plot, seeing how the cost gets closer to the global minimum as gradient descent finds better and better values for the parameters W and B. So remember that to do the optional lab, you just need to read and run this code. You won't need to write any code yourself. And I hope you take a few moments to do that and also become familiar with the gradient descent code because this will help you to implement this and similar algorithms in the future yourself. Thanks for sticking with me through the end of this last video for the first week and congratulations for making it all the way here. You're on your way to becoming a machine learning person. In addition to the optional labs, if you haven't done so yet, I hope you also check out the practice quizzes, which are a nice way that you can double check your own understanding of the concepts. It's also totally fine if you don't get them all right the first time. And you can also take the quizzes multiple times until you get the score that you want. You now know how to implement linear regression with one variable. And that brings us to the close of this week. Next week, we'll learn to make linear regression much more powerful. Instead of one feature like size of a house, you learn how to get it to work with lots of features. You also learn how to get it to fit nonlinear curves. These improvements will make the algorithm much more useful and valuable. Lastly, we'll also go over some practical tips that will really help for getting linear regression to work on practical applications. I'm really happy to have you here with me in this class and I look forward to seeing you next week.
[{"start": 0.0, "end": 6.4, "text": " Let's see what happens when you run gradient descent for linear regression."}, {"start": 6.4, "end": 8.88, "text": " Let's go see the algorithm in action."}, {"start": 8.88, "end": 15.56, "text": " Here's a plot of the model and data on the upper left, and a contour plot of the cost"}, {"start": 15.56, "end": 18.32, "text": " function on the upper right."}, {"start": 18.32, "end": 23.64, "text": " And at the bottom is the surface plot of the same cost function."}, {"start": 23.64, "end": 29.64, "text": " Often W and B will both be initialized to zero, but for this demonstration, let's initialize"}, {"start": 29.64, "end": 36.08, "text": " W to be equal to negative 0.1 and B to be 900."}, {"start": 36.08, "end": 44.760000000000005, "text": " So this corresponds to f of x equals negative 0.1 x plus 900."}, {"start": 44.760000000000005, "end": 50.56, "text": " Now if we take one step using gradient descent, we end up going from this point of the cost"}, {"start": 50.56, "end": 57.8, "text": " function out here to this point just down and to the right."}, {"start": 57.8, "end": 64.39999999999999, "text": " And notice that the straight line fit has also changed a bit."}, {"start": 64.39999999999999, "end": 67.03999999999999, "text": " Let's take another step."}, {"start": 67.03999999999999, "end": 73.6, "text": " The cost function has now moved to this third point, and again the function f of x has also"}, {"start": 73.6, "end": 76.08, "text": " changed a bit."}, {"start": 76.08, "end": 82.66, "text": " As you take more of these steps, the cost is decreasing at each update, so the parameters"}, {"start": 82.66, "end": 88.52, "text": " W and B are following this trajectory."}, {"start": 88.52, "end": 94.2, "text": " And if you look on the left, you get this corresponding straight line fit that, you"}, {"start": 94.2, "end": 100.42, "text": " know, fits the data better and better until we've reached the global minimum."}, {"start": 100.42, "end": 106.6, "text": " The global minimum corresponds to this straight line fit, which is a relatively good fit to"}, {"start": 106.6, "end": 107.6, "text": " the data."}, {"start": 107.6, "end": 110.44, "text": " I mean, isn't that cool?"}, {"start": 110.44, "end": 117.0, "text": " And so that's gradient descent, and we're going to use this to fit a model to the housing"}, {"start": 117.0, "end": 118.67999999999999, "text": " data."}, {"start": 118.67999999999999, "end": 125.42, "text": " And you can now use this f of x model to predict the price of your client's house or anyone"}, {"start": 125.42, "end": 126.88, "text": " else's house."}, {"start": 126.88, "end": 133.96, "text": " For instance, if your friend's house size is 1250 square feet, you can now read off"}, {"start": 133.96, "end": 141.60000000000002, "text": " the value and predict that maybe they could get, I don't know, $250,000 for the house."}, {"start": 141.60000000000002, "end": 147.76000000000002, "text": " To be more precise, this gradient descent process is called batch gradient descent."}, {"start": 147.76000000000002, "end": 154.0, "text": " The term batch gradient descent refers to the fact that on every step of gradient descent,"}, {"start": 154.0, "end": 161.4, "text": " we're looking at all of the training examples instead of just a subset of the training data."}, {"start": 161.4, "end": 167.72, "text": " So in computing gradient descent, when computing derivatives, we're computing the sum from"}, {"start": 167.72, "end": 175.92000000000002, "text": " i equals one to m, and batch gradient descent is looking at the entire batch of training"}, {"start": 175.92000000000002, "end": 179.04000000000002, "text": " examples at each update."}, {"start": 179.04000000000002, "end": 183.76, "text": " I know that batch gradient descent may not be the most intuitive name, but this is what"}, {"start": 183.76, "end": 187.16, "text": " people in the machine learning community call it."}, {"start": 187.16, "end": 193.72, "text": " If you've heard of the newsletter, The Batch, that's published by deeplearning.ai, the newsletter,"}, {"start": 193.72, "end": 198.64, "text": " The Batch, was also named for this concept in machine learning."}, {"start": 198.64, "end": 202.88, "text": " And then it turns out that there are other versions of gradient descent that do not look"}, {"start": 202.88, "end": 208.44, "text": " at the entire training set, but instead looks at smaller subsets of the training data at"}, {"start": 208.44, "end": 210.18, "text": " each update step."}, {"start": 210.18, "end": 214.72, "text": " But we'll use batch gradient descent for linear regression."}, {"start": 214.72, "end": 216.8, "text": " So that's it for linear regression."}, {"start": 216.8, "end": 220.8, "text": " Congratulations on getting through your first machine learning model."}, {"start": 220.8, "end": 226.20000000000002, "text": " I hope you go and celebrate or, I don't know, maybe take a nap in your hammock."}, {"start": 226.20000000000002, "end": 231.64000000000001, "text": " In the optional lab that follows this video, you see a review of the gradient descent algorithm"}, {"start": 231.64000000000001, "end": 234.68, "text": " as well as how to implement it in code."}, {"start": 234.68, "end": 241.68, "text": " You also see a plot that shows how the cost decreases as you continue training more iterations."}, {"start": 241.68, "end": 247.20000000000002, "text": " And you also see a contour plot, seeing how the cost gets closer to the global minimum"}, {"start": 247.20000000000002, "end": 253.6, "text": " as gradient descent finds better and better values for the parameters W and B."}, {"start": 253.6, "end": 259.54, "text": " So remember that to do the optional lab, you just need to read and run this code."}, {"start": 259.54, "end": 262.44, "text": " You won't need to write any code yourself."}, {"start": 262.44, "end": 267.44, "text": " And I hope you take a few moments to do that and also become familiar with the gradient"}, {"start": 267.44, "end": 274.36, "text": " descent code because this will help you to implement this and similar algorithms in the"}, {"start": 274.36, "end": 276.8, "text": " future yourself."}, {"start": 276.8, "end": 280.64, "text": " Thanks for sticking with me through the end of this last video for the first week and"}, {"start": 280.64, "end": 283.36, "text": " congratulations for making it all the way here."}, {"start": 283.36, "end": 287.4, "text": " You're on your way to becoming a machine learning person."}, {"start": 287.4, "end": 292.76, "text": " In addition to the optional labs, if you haven't done so yet, I hope you also check out the"}, {"start": 292.76, "end": 297.08, "text": " practice quizzes, which are a nice way that you can double check your own understanding"}, {"start": 297.08, "end": 298.08, "text": " of the concepts."}, {"start": 298.08, "end": 302.64, "text": " It's also totally fine if you don't get them all right the first time."}, {"start": 302.64, "end": 308.09999999999997, "text": " And you can also take the quizzes multiple times until you get the score that you want."}, {"start": 308.09999999999997, "end": 312.64, "text": " You now know how to implement linear regression with one variable."}, {"start": 312.64, "end": 316.15999999999997, "text": " And that brings us to the close of this week."}, {"start": 316.15999999999997, "end": 320.32, "text": " Next week, we'll learn to make linear regression much more powerful."}, {"start": 320.32, "end": 324.97999999999996, "text": " Instead of one feature like size of a house, you learn how to get it to work with lots"}, {"start": 324.97999999999996, "end": 326.24, "text": " of features."}, {"start": 326.24, "end": 330.28000000000003, "text": " You also learn how to get it to fit nonlinear curves."}, {"start": 330.28000000000003, "end": 334.36, "text": " These improvements will make the algorithm much more useful and valuable."}, {"start": 334.36, "end": 339.76, "text": " Lastly, we'll also go over some practical tips that will really help for getting linear"}, {"start": 339.76, "end": 343.28000000000003, "text": " regression to work on practical applications."}, {"start": 343.28000000000003, "end": 346.76, "text": " I'm really happy to have you here with me in this class and I look forward to seeing"}, {"start": 346.76, "end": 357.44, "text": " you next week."}]