1.14 Machine Learning Overview | Visualizing examples --[Machine Learning | Andrew Ng]
 Let's look at some more visualizations of w and b. Here's one example. Over here, you have a particular point on the graph J. For this point, w equals about negative 0.15 and b equals about 800. So this point corresponds to one pair of values for w and b that use a particular cost J. And in fact, this particular pair of values for w and b corresponds to this function f of x, which is this line that you can see on the left. This line intersects the vertical axis at 800 because b equals 800 and the slope of the line is negative 0.15 because w equals negative 0.15. Now if you look at the data points in the training set, you may notice that this line is not a good fit to the data. For this function f of x, with these values of w and b, many of the predictions for the value of y are quite far from the actual target value of y that is in the training data. Since this line is not a good fit, if you look at the graph of J, the cost of this line is out here, which is pretty far from the minimum. This is a pretty high cost because this choice of w and b is just not that good a fit to the training set. Now let's look at another example with a different choice of w and b. Now here is another function that is still not a great fit for the data, but maybe slightly less bad. So this point here represents the cost for this particular pair of w and b that creates that line. The value of w is equal to 0 and the value of b is about 360. This pair of parameters corresponds to this function, which is a flat line because f of x equals 0 times x plus 360. I hope that makes sense. Let's look at yet another example. Here's one more choice for w and b, and with these values, you end up with this line f of x. Again, not a great fit to the data. It is actually further away from the minimum compared to the previous example. And remember that the minimum is at the center of that smallest ellipse. This example, if you look at f of x on the left, this looks like a pretty good fit to the training set. You can see on the right, this point representing the cost is very close to the center of the small ellipse. It's not quite exactly the minimum, but it's pretty close. For this value of w and b, you get this line f of x. You can see that if you measure the vertical distances between the data points and the predicted values on the straight line, you get the error for each data point. The sum of squared errors for all of these data points is pretty close to the minimum possible sum of squared errors among all possible straight line fits. I hope that by looking at these figures, you can get a better sense of how different choices of the parameters affect the line f of x and how this corresponds to different values for the cost j. And hopefully, you can see how the better fit lines correspond to points on the graph of j that are closer to the minimum possible cost for this cost function j of w and b. In the optional lab that follows this video, you get to run some code. And remember, all of the code is given, so you just need to hit shift enter to run it and take a look at it. And the lab will show you how the cost function is implemented in code. And given a small training set and different choices for the parameters, you'll be able to see how the cost varies depending on how well the model fits the data. In the optional lab, you also can play with an interactive contour plot. Check this out. You can use your mouse cursor to click anywhere on the contour plot, and you're going to see the straight line defined by the values you chose for the parameters w and b. You see a dot appear also on the 3D surface plot showing the cost. Finally, the optional lab also has a 3D surface plot that you can manually rotate and spin around using your mouse cursor to take a better look at what the cost function looks like. I hope you enjoyed playing with the optional lab. Now, in linear regression, rather than having to manually try to read the contour plot for the best value for w and b, which isn't really a good procedure and also won't work once we get to more complex machine learning models, what you really want is an efficient algorithm that you can write in code for automatically finding the values of parameters w and b that gives you the best fit line that minimizes the cost function j. There is an algorithm for doing this called gradient descent. This algorithm is one of the most important algorithms in machine learning. Gradient descent and variations on gradient descent are used to train not just linear regression, but some of the biggest and most complex models in all of AI. So, let's go to the next video to dive into this really important algorithm called gradient descent.
[{"start": 0.0, "end": 7.6000000000000005, "text": " Let's look at some more visualizations of w and b."}, {"start": 7.6000000000000005, "end": 9.8, "text": " Here's one example."}, {"start": 9.8, "end": 17.32, "text": " Over here, you have a particular point on the graph J. For this point, w equals about"}, {"start": 17.32, "end": 22.82, "text": " negative 0.15 and b equals about 800."}, {"start": 22.82, "end": 30.36, "text": " So this point corresponds to one pair of values for w and b that use a particular cost J."}, {"start": 30.36, "end": 36.4, "text": " And in fact, this particular pair of values for w and b corresponds to this function f"}, {"start": 36.4, "end": 41.22, "text": " of x, which is this line that you can see on the left."}, {"start": 41.22, "end": 48.519999999999996, "text": " This line intersects the vertical axis at 800 because b equals 800 and the slope of"}, {"start": 48.52, "end": 54.56, "text": " the line is negative 0.15 because w equals negative 0.15."}, {"start": 54.56, "end": 58.92, "text": " Now if you look at the data points in the training set, you may notice that this line"}, {"start": 58.92, "end": 61.800000000000004, "text": " is not a good fit to the data."}, {"start": 61.800000000000004, "end": 69.04, "text": " For this function f of x, with these values of w and b, many of the predictions for the"}, {"start": 69.04, "end": 76.68, "text": " value of y are quite far from the actual target value of y that is in the training data."}, {"start": 76.68, "end": 82.60000000000001, "text": " Since this line is not a good fit, if you look at the graph of J, the cost of this line"}, {"start": 82.60000000000001, "end": 87.08000000000001, "text": " is out here, which is pretty far from the minimum."}, {"start": 87.08000000000001, "end": 92.64000000000001, "text": " This is a pretty high cost because this choice of w and b is just not that good a fit to"}, {"start": 92.64000000000001, "end": 95.48, "text": " the training set."}, {"start": 95.48, "end": 102.2, "text": " Now let's look at another example with a different choice of w and b."}, {"start": 102.2, "end": 107.48, "text": " Now here is another function that is still not a great fit for the data, but maybe slightly"}, {"start": 107.48, "end": 109.16, "text": " less bad."}, {"start": 109.16, "end": 115.4, "text": " So this point here represents the cost for this particular pair of w and b that creates"}, {"start": 115.4, "end": 117.4, "text": " that line."}, {"start": 117.4, "end": 124.32000000000001, "text": " The value of w is equal to 0 and the value of b is about 360."}, {"start": 124.32000000000001, "end": 129.44, "text": " This pair of parameters corresponds to this function, which is a flat line because f of"}, {"start": 129.44, "end": 133.84, "text": " x equals 0 times x plus 360."}, {"start": 133.84, "end": 136.44, "text": " I hope that makes sense."}, {"start": 136.44, "end": 139.28, "text": " Let's look at yet another example."}, {"start": 139.28, "end": 144.84, "text": " Here's one more choice for w and b, and with these values, you end up with this line f"}, {"start": 144.84, "end": 145.84, "text": " of x."}, {"start": 145.84, "end": 147.68, "text": " Again, not a great fit to the data."}, {"start": 147.68, "end": 153.28, "text": " It is actually further away from the minimum compared to the previous example."}, {"start": 153.28, "end": 159.12, "text": " And remember that the minimum is at the center of that smallest ellipse."}, {"start": 159.12, "end": 165.64000000000001, "text": " This example, if you look at f of x on the left, this looks like a pretty good fit to"}, {"start": 165.64000000000001, "end": 167.16, "text": " the training set."}, {"start": 167.16, "end": 175.28, "text": " You can see on the right, this point representing the cost is very close to the center of the"}, {"start": 175.28, "end": 176.28, "text": " small ellipse."}, {"start": 176.28, "end": 180.4, "text": " It's not quite exactly the minimum, but it's pretty close."}, {"start": 180.4, "end": 187.08, "text": " For this value of w and b, you get this line f of x."}, {"start": 187.08, "end": 191.52, "text": " You can see that if you measure the vertical distances between the data points and the"}, {"start": 191.52, "end": 198.88000000000002, "text": " predicted values on the straight line, you get the error for each data point."}, {"start": 198.88000000000002, "end": 204.92000000000002, "text": " The sum of squared errors for all of these data points is pretty close to the minimum"}, {"start": 204.92000000000002, "end": 210.92000000000002, "text": " possible sum of squared errors among all possible straight line fits."}, {"start": 210.92000000000002, "end": 215.92000000000002, "text": " I hope that by looking at these figures, you can get a better sense of how different choices"}, {"start": 215.92, "end": 223.28, "text": " of the parameters affect the line f of x and how this corresponds to different values for"}, {"start": 223.28, "end": 225.83999999999997, "text": " the cost j."}, {"start": 225.83999999999997, "end": 232.04, "text": " And hopefully, you can see how the better fit lines correspond to points on the graph"}, {"start": 232.04, "end": 241.83999999999997, "text": " of j that are closer to the minimum possible cost for this cost function j of w and b."}, {"start": 241.84, "end": 247.32, "text": " In the optional lab that follows this video, you get to run some code."}, {"start": 247.32, "end": 251.92000000000002, "text": " And remember, all of the code is given, so you just need to hit shift enter to run it"}, {"start": 251.92000000000002, "end": 253.76, "text": " and take a look at it."}, {"start": 253.76, "end": 258.88, "text": " And the lab will show you how the cost function is implemented in code."}, {"start": 258.88, "end": 263.94, "text": " And given a small training set and different choices for the parameters, you'll be able"}, {"start": 263.94, "end": 269.92, "text": " to see how the cost varies depending on how well the model fits the data."}, {"start": 269.92, "end": 274.04, "text": " In the optional lab, you also can play with an interactive contour plot."}, {"start": 274.04, "end": 275.52000000000004, "text": " Check this out."}, {"start": 275.52000000000004, "end": 280.44, "text": " You can use your mouse cursor to click anywhere on the contour plot, and you're going to see"}, {"start": 280.44, "end": 285.84000000000003, "text": " the straight line defined by the values you chose for the parameters w and b."}, {"start": 285.84000000000003, "end": 291.68, "text": " You see a dot appear also on the 3D surface plot showing the cost."}, {"start": 291.68, "end": 298.6, "text": " Finally, the optional lab also has a 3D surface plot that you can manually rotate and spin"}, {"start": 298.6, "end": 304.92, "text": " around using your mouse cursor to take a better look at what the cost function looks like."}, {"start": 304.92, "end": 307.48, "text": " I hope you enjoyed playing with the optional lab."}, {"start": 307.48, "end": 313.40000000000003, "text": " Now, in linear regression, rather than having to manually try to read the contour plot for"}, {"start": 313.40000000000003, "end": 318.44, "text": " the best value for w and b, which isn't really a good procedure and also won't work once"}, {"start": 318.44, "end": 324.40000000000003, "text": " we get to more complex machine learning models, what you really want is an efficient algorithm"}, {"start": 324.4, "end": 329.79999999999995, "text": " that you can write in code for automatically finding the values of parameters w and b that"}, {"start": 329.79999999999995, "end": 335.35999999999996, "text": " gives you the best fit line that minimizes the cost function j."}, {"start": 335.35999999999996, "end": 339.08, "text": " There is an algorithm for doing this called gradient descent."}, {"start": 339.08, "end": 343.59999999999997, "text": " This algorithm is one of the most important algorithms in machine learning."}, {"start": 343.59999999999997, "end": 348.35999999999996, "text": " Gradient descent and variations on gradient descent are used to train not just linear"}, {"start": 348.35999999999996, "end": 353.15999999999997, "text": " regression, but some of the biggest and most complex models in all of AI."}, {"start": 353.16, "end": 359.08000000000004, "text": " So, let's go to the next video to dive into this really important algorithm called gradient"}, {"start": 359.08, "end": 383.88, "text": " descent."}]