6.4 Bias and variance | Diagnosing bias and variance -[Machine Learning | Andrew Ng]
 The typical workflow of developing a machine learning system is that you have an idea and you train a model and you almost always find that it doesn't work as well as you wish yet. When I'm training machine learning model, it pretty much never works that well the first time. And so key to the process of building machine learning system is how to decide what to do next in order to improve this performance. I've found across many different applications that looking at the bias and variance of a learning algorithm gives you very good guidance on what to try next. Let's take a look at what this means. You might remember this example from the first course on linear regression, where given this data set, if you were to fill a straight line to it, it doesn't do that well. And we said that this algorithm has high bias or that it underfits this data set. Or if you were to fit a four-folder polynomial, then it has high variance or it overfits. And in the middle, if you fit a quadratic polynomial, then it looks pretty good. And I said that was just right. Because this is a problem with just a single feature x, we could plot the function f and look at it like this. But if you had more features, you can't plot f and visualize whether it's doing well as easily. So instead of trying to look at plots like this, a more systematic way to diagnose or to find out if your algorithm has high bias or high variance will be to look at the performance of your algorithm on the training set and on the cross validation set. In particular, let's look at the example on the left. If you were to compute J train, how well does the algorithm do on the training set? Not that well. So let's say J train here would be high because there are actually pretty large errors between the examples and the actual predictions of the model. And how about Jcv? So Jcv would be if you had a few new examples, maybe examples like that, that the algorithm had not previously seen. And here, the algorithm also doesn't do that well on examples that it had not previously seen. So Jcv would also be high. And one characteristic of an algorithm with high bias, something that is underfitting is that it's not even doing that well on the training set. And so when J train is high, that gives you a strong indicator that this algorithm has high bias. Let's now look at the example on the right. If you were to compute J train, how well is this doing on the training set? Well, it's actually doing great on the training set, fits the training data really well. So J train here will be low. But if you were to evaluate this model on other houses not in the training set, then you find that Jcv, the cross validation error will be quite high. And so a characteristic signature or characteristic cue that your algorithm has high variance will be of Jcv is much higher than J train. In other words, it does much better on data it has seen than on data it has not seen. And this turns out to be a strong indicator that your algorithm has high variance. And again, the point of what we're doing is that by computing J train and Jcv and seeing if J train is high, or if Jcv is much higher than J train, this gives you a sense even if you can't plot to function f of whether your algorithm has high bias or high variance. And finally, the case in the middle, if you look at J train is pretty low since it's doing quite well on the training set. And if you were to look at a few new examples like those from say your cross validation set, you find that Jcv is also pretty low. And so J train not being too high, indicates this doesn't have a high bias problem. And Jcv not being much worse than J train, this indicates that it doesn't have a high variance problem either, which is why this model, the quadratic model seems to be a pretty good one for this application. Let me share with you another view of bias and variance. So to summarize when d equals one for a linear polynomial, J train was high and Jcv was high. When d equals four, J train was low, but Jcv is high. And when d equals two, both were pretty low. Let's now take a different view on bias and variance. And in particular, on the next slide, I'd like to show you how J train and Jcv vary as a function of the degree of the polynomial you're fitting. So let me draw a figure where the horizontal axis of this figure will be the degree of polynomial that we're fitting to the data. Over on the left will correspond to a small value of d, like d equals one, which corresponds to a fitting straight line. And over to the right will correspond to say d equals four or even higher values of d, where we're fitting this high order polynomial. So if you were to plot J train of Wb as a function of degree of polynomial, what you find is that as you fit a higher and higher degree polynomial, here I'm assuming we're not using regularization, but as you fit a higher and higher order polynomial, the training error will tend to go down because when you have a very simple linear function, it doesn't fit the training data that well. When you fit a quadratic function or a third order polynomial or a fourth order polynomial, it fits the training data better and better. So as the degree of polynomial increases, J train will typically go down. Next, let's look at Jcv, which is how well does it do on data that it did not get to fit to? What we saw was when d equals one, when the degree of polynomial was very low, Jcv was pretty high because it underfit, so it didn't do well on the cross validation set. And here on the right as well, when the degree of polynomial is very large, say four, it doesn't do well on the cross validation set either and so is also high. But if d was in between, say a second order polynomial, then it actually did much better. And so if you were to vary the degree of polynomial, you'd actually get a curve that looks like this, which comes down and then goes back up. Where if the degree of polynomial is too low, it underfits and so doesn't do well on the cross validation set. If it is too high, it overfits and also doesn't do well on the cross validation set. And there's only if it's somewhere in the middle that is just right, which is why the second order polynomial in our example ends up with a lower cross validation error and neither high bias nor high variance. So to summarize, how do you diagnose bias and variance in your learning algorithm? If your learning algorithm has high bias or has underfit data, the key indicator will be if Jtrain is high. And so that corresponds to this leftmost portion of the curve, which is where Jtrain is high. And usually you have Jtrain and JCV will be close to each other. And how do you diagnose if you have high variance? Well the key indicator for high variance will be if JCV is much greater than Jtrain. This double greater than sign in math refers to much greater than. So this is greater and this means much greater. And this rightmost portion of the plot is where JCV is much greater than Jtrain. And usually Jtrain will be pretty low, but the key indicator is whether JCV is much greater than Jtrain. And that's what happens when we had fit a very high order polynomial to this small dataset. And even though we've just seen bias and variance, it turns out in some cases it's possible to simultaneously have high bias and have high variance. You won't see this happen that much for linear regression, but it turns out that if you're training a neural network, there are some applications where unfortunately you have high bias and high variance. And one way to recognize that situation will be if Jtrain is high, so you're not doing that well on the training set, but even worse the cross validation error is again even much larger than the training set. The notion of high bias and high variance, it doesn't really happen for linear models applied to 1D, but to give intuition about what it looks like, it would be as if for part of the input you had a very complicated model that overfit, so it overfits to part of the input. But then for some reason for other parts of the input, it doesn't even fit the training data well, and so it underfits for part of the input. In this example, which looks artificial because it's a single feature input, we fit the training set really well and we overfit in part of the input, and we don't even fit the training data well and we underfit in part of the input. And that's how in some applications you can unfortunately end up with both high bias and high variance. And the indicator for that will be if the algorithm does poorly on the training set and it even does much worse than on the training set. For most learning applications, you probably have primarily a high bias or a high variance problem rather than both at the same time, but it is possible sometimes that both are the same time. So I know that there's a lot to process, there are a lot of concepts on the slides, but the key takeaways are high bias means it's not even doing well on the training set, and high variance means it does much worse on the cross-validation set than the training set. Whenever I'm training a machine learning algorithm, I will almost always try to figure out to what extent the algorithm has a high bias or underfitting versus a high variance or an overfitting problem. And this will give good guidance, as we'll see later this week, on how you can improve the performance of the algorithm. But first, let's take a look at how regularization affects the bias and variance of a learning algorithm, because that will help you better understand when you should use regularization. Let's take a look at that in the next video.
[{"start": 0.0, "end": 8.2, "text": " The typical workflow of developing a machine learning system is that you have an idea and"}, {"start": 8.2, "end": 14.200000000000001, "text": " you train a model and you almost always find that it doesn't work as well as you wish yet."}, {"start": 14.200000000000001, "end": 17.92, "text": " When I'm training machine learning model, it pretty much never works that well the first"}, {"start": 17.92, "end": 18.92, "text": " time."}, {"start": 18.92, "end": 23.52, "text": " And so key to the process of building machine learning system is how to decide what to do"}, {"start": 23.52, "end": 26.12, "text": " next in order to improve this performance."}, {"start": 26.12, "end": 31.04, "text": " I've found across many different applications that looking at the bias and variance of a"}, {"start": 31.04, "end": 35.36, "text": " learning algorithm gives you very good guidance on what to try next."}, {"start": 35.36, "end": 38.120000000000005, "text": " Let's take a look at what this means."}, {"start": 38.120000000000005, "end": 45.52, "text": " You might remember this example from the first course on linear regression, where given this"}, {"start": 45.52, "end": 49.66, "text": " data set, if you were to fill a straight line to it, it doesn't do that well."}, {"start": 49.66, "end": 56.08, "text": " And we said that this algorithm has high bias or that it underfits this data set."}, {"start": 56.08, "end": 64.2, "text": " Or if you were to fit a four-folder polynomial, then it has high variance or it overfits."}, {"start": 64.2, "end": 68.88, "text": " And in the middle, if you fit a quadratic polynomial, then it looks pretty good."}, {"start": 68.88, "end": 71.46, "text": " And I said that was just right."}, {"start": 71.46, "end": 76.12, "text": " Because this is a problem with just a single feature x, we could plot the function f and"}, {"start": 76.12, "end": 77.6, "text": " look at it like this."}, {"start": 77.6, "end": 83.03999999999999, "text": " But if you had more features, you can't plot f and visualize whether it's doing well as"}, {"start": 83.03999999999999, "end": 84.8, "text": " easily."}, {"start": 84.8, "end": 91.47999999999999, "text": " So instead of trying to look at plots like this, a more systematic way to diagnose or"}, {"start": 91.47999999999999, "end": 97.52, "text": " to find out if your algorithm has high bias or high variance will be to look at the performance"}, {"start": 97.52, "end": 102.75999999999999, "text": " of your algorithm on the training set and on the cross validation set."}, {"start": 102.75999999999999, "end": 105.6, "text": " In particular, let's look at the example on the left."}, {"start": 105.6, "end": 112.16, "text": " If you were to compute J train, how well does the algorithm do on the training set?"}, {"start": 112.16, "end": 113.16, "text": " Not that well."}, {"start": 113.16, "end": 118.64, "text": " So let's say J train here would be high because there are actually pretty large errors between"}, {"start": 118.64, "end": 123.2, "text": " the examples and the actual predictions of the model."}, {"start": 123.2, "end": 124.92, "text": " And how about Jcv?"}, {"start": 124.92, "end": 133.4, "text": " So Jcv would be if you had a few new examples, maybe examples like that, that the algorithm"}, {"start": 133.4, "end": 136.16, "text": " had not previously seen."}, {"start": 136.16, "end": 141.44, "text": " And here, the algorithm also doesn't do that well on examples that it had not previously"}, {"start": 141.44, "end": 142.44, "text": " seen."}, {"start": 142.44, "end": 145.88, "text": " So Jcv would also be high."}, {"start": 145.88, "end": 150.4, "text": " And one characteristic of an algorithm with high bias, something that is underfitting"}, {"start": 150.4, "end": 154.16, "text": " is that it's not even doing that well on the training set."}, {"start": 154.16, "end": 159.0, "text": " And so when J train is high, that gives you a strong indicator that this algorithm has"}, {"start": 159.0, "end": 160.52, "text": " high bias."}, {"start": 160.52, "end": 163.2, "text": " Let's now look at the example on the right."}, {"start": 163.2, "end": 167.44, "text": " If you were to compute J train, how well is this doing on the training set?"}, {"start": 167.44, "end": 172.2, "text": " Well, it's actually doing great on the training set, fits the training data really well."}, {"start": 172.2, "end": 174.64, "text": " So J train here will be low."}, {"start": 174.64, "end": 181.48, "text": " But if you were to evaluate this model on other houses not in the training set, then"}, {"start": 181.48, "end": 187.94, "text": " you find that Jcv, the cross validation error will be quite high."}, {"start": 187.94, "end": 193.76, "text": " And so a characteristic signature or characteristic cue that your algorithm has high variance"}, {"start": 193.76, "end": 198.67999999999998, "text": " will be of Jcv is much higher than J train."}, {"start": 198.68, "end": 203.28, "text": " In other words, it does much better on data it has seen than on data it has not seen."}, {"start": 203.28, "end": 208.56, "text": " And this turns out to be a strong indicator that your algorithm has high variance."}, {"start": 208.56, "end": 215.32, "text": " And again, the point of what we're doing is that by computing J train and Jcv and seeing"}, {"start": 215.32, "end": 221.76000000000002, "text": " if J train is high, or if Jcv is much higher than J train, this gives you a sense even"}, {"start": 221.76, "end": 229.32, "text": " if you can't plot to function f of whether your algorithm has high bias or high variance."}, {"start": 229.32, "end": 236.67999999999998, "text": " And finally, the case in the middle, if you look at J train is pretty low since it's doing"}, {"start": 236.67999999999998, "end": 239.28, "text": " quite well on the training set."}, {"start": 239.28, "end": 245.44, "text": " And if you were to look at a few new examples like those from say your cross validation"}, {"start": 245.44, "end": 249.6, "text": " set, you find that Jcv is also pretty low."}, {"start": 249.6, "end": 257.64, "text": " And so J train not being too high, indicates this doesn't have a high bias problem."}, {"start": 257.64, "end": 262.76, "text": " And Jcv not being much worse than J train, this indicates that it doesn't have a high"}, {"start": 262.76, "end": 268.96, "text": " variance problem either, which is why this model, the quadratic model seems to be a pretty"}, {"start": 268.96, "end": 271.64, "text": " good one for this application."}, {"start": 271.64, "end": 276.36, "text": " Let me share with you another view of bias and variance."}, {"start": 276.36, "end": 284.66, "text": " So to summarize when d equals one for a linear polynomial, J train was high and Jcv was high."}, {"start": 284.66, "end": 289.66, "text": " When d equals four, J train was low, but Jcv is high."}, {"start": 289.66, "end": 293.36, "text": " And when d equals two, both were pretty low."}, {"start": 293.36, "end": 296.2, "text": " Let's now take a different view on bias and variance."}, {"start": 296.2, "end": 302.52000000000004, "text": " And in particular, on the next slide, I'd like to show you how J train and Jcv vary"}, {"start": 302.52, "end": 307.28, "text": " as a function of the degree of the polynomial you're fitting."}, {"start": 307.28, "end": 311.64, "text": " So let me draw a figure where the horizontal axis of this figure will be the degree of"}, {"start": 311.64, "end": 316.28, "text": " polynomial that we're fitting to the data."}, {"start": 316.28, "end": 321.59999999999997, "text": " Over on the left will correspond to a small value of d, like d equals one, which corresponds"}, {"start": 321.59999999999997, "end": 323.44, "text": " to a fitting straight line."}, {"start": 323.44, "end": 329.91999999999996, "text": " And over to the right will correspond to say d equals four or even higher values of d,"}, {"start": 329.92, "end": 333.72, "text": " where we're fitting this high order polynomial."}, {"start": 333.72, "end": 342.64000000000004, "text": " So if you were to plot J train of Wb as a function of degree of polynomial, what you"}, {"start": 342.64000000000004, "end": 349.12, "text": " find is that as you fit a higher and higher degree polynomial, here I'm assuming we're"}, {"start": 349.12, "end": 354.48, "text": " not using regularization, but as you fit a higher and higher order polynomial, the training"}, {"start": 354.48, "end": 359.64, "text": " error will tend to go down because when you have a very simple linear function, it doesn't"}, {"start": 359.64, "end": 361.71999999999997, "text": " fit the training data that well."}, {"start": 361.71999999999997, "end": 367.52, "text": " When you fit a quadratic function or a third order polynomial or a fourth order polynomial,"}, {"start": 367.52, "end": 370.44, "text": " it fits the training data better and better."}, {"start": 370.44, "end": 376.96, "text": " So as the degree of polynomial increases, J train will typically go down."}, {"start": 376.96, "end": 386.96, "text": " Next, let's look at Jcv, which is how well does it do on data that it did not get to"}, {"start": 386.96, "end": 388.15999999999997, "text": " fit to?"}, {"start": 388.16, "end": 394.64000000000004, "text": " What we saw was when d equals one, when the degree of polynomial was very low, Jcv was"}, {"start": 394.64000000000004, "end": 400.40000000000003, "text": " pretty high because it underfit, so it didn't do well on the cross validation set."}, {"start": 400.40000000000003, "end": 406.76000000000005, "text": " And here on the right as well, when the degree of polynomial is very large, say four, it"}, {"start": 406.76000000000005, "end": 412.54, "text": " doesn't do well on the cross validation set either and so is also high."}, {"start": 412.54, "end": 418.14000000000004, "text": " But if d was in between, say a second order polynomial, then it actually did much better."}, {"start": 418.14, "end": 423.47999999999996, "text": " And so if you were to vary the degree of polynomial, you'd actually get a curve that looks like"}, {"start": 423.47999999999996, "end": 427.15999999999997, "text": " this, which comes down and then goes back up."}, {"start": 427.15999999999997, "end": 433.08, "text": " Where if the degree of polynomial is too low, it underfits and so doesn't do well on the"}, {"start": 433.08, "end": 434.32, "text": " cross validation set."}, {"start": 434.32, "end": 440.4, "text": " If it is too high, it overfits and also doesn't do well on the cross validation set."}, {"start": 440.4, "end": 445.4, "text": " And there's only if it's somewhere in the middle that is just right, which is why the"}, {"start": 445.4, "end": 450.67999999999995, "text": " second order polynomial in our example ends up with a lower cross validation error and"}, {"start": 450.67999999999995, "end": 454.35999999999996, "text": " neither high bias nor high variance."}, {"start": 454.35999999999996, "end": 460.97999999999996, "text": " So to summarize, how do you diagnose bias and variance in your learning algorithm?"}, {"start": 460.97999999999996, "end": 466.0, "text": " If your learning algorithm has high bias or has underfit data, the key indicator will"}, {"start": 466.0, "end": 469.15999999999997, "text": " be if Jtrain is high."}, {"start": 469.15999999999997, "end": 474.52, "text": " And so that corresponds to this leftmost portion of the curve, which is where Jtrain is high."}, {"start": 474.52, "end": 479.71999999999997, "text": " And usually you have Jtrain and JCV will be close to each other."}, {"start": 479.71999999999997, "end": 482.71999999999997, "text": " And how do you diagnose if you have high variance?"}, {"start": 482.71999999999997, "end": 489.03999999999996, "text": " Well the key indicator for high variance will be if JCV is much greater than Jtrain."}, {"start": 489.03999999999996, "end": 492.79999999999995, "text": " This double greater than sign in math refers to much greater than."}, {"start": 492.79999999999995, "end": 496.96, "text": " So this is greater and this means much greater."}, {"start": 496.96, "end": 503.47999999999996, "text": " And this rightmost portion of the plot is where JCV is much greater than Jtrain."}, {"start": 503.48, "end": 510.0, "text": " And usually Jtrain will be pretty low, but the key indicator is whether JCV is much greater"}, {"start": 510.0, "end": 511.64000000000004, "text": " than Jtrain."}, {"start": 511.64000000000004, "end": 517.72, "text": " And that's what happens when we had fit a very high order polynomial to this small dataset."}, {"start": 517.72, "end": 523.94, "text": " And even though we've just seen bias and variance, it turns out in some cases it's possible to"}, {"start": 523.94, "end": 528.6, "text": " simultaneously have high bias and have high variance."}, {"start": 528.6, "end": 533.0, "text": " You won't see this happen that much for linear regression, but it turns out that if you're"}, {"start": 533.0, "end": 536.96, "text": " training a neural network, there are some applications where unfortunately you have"}, {"start": 536.96, "end": 539.76, "text": " high bias and high variance."}, {"start": 539.76, "end": 545.48, "text": " And one way to recognize that situation will be if Jtrain is high, so you're not doing"}, {"start": 545.48, "end": 550.88, "text": " that well on the training set, but even worse the cross validation error is again even much"}, {"start": 550.88, "end": 554.0, "text": " larger than the training set."}, {"start": 554.0, "end": 559.16, "text": " The notion of high bias and high variance, it doesn't really happen for linear models"}, {"start": 559.16, "end": 566.0, "text": " applied to 1D, but to give intuition about what it looks like, it would be as if for"}, {"start": 566.0, "end": 573.4399999999999, "text": " part of the input you had a very complicated model that overfit, so it overfits to part"}, {"start": 573.4399999999999, "end": 574.56, "text": " of the input."}, {"start": 574.56, "end": 579.68, "text": " But then for some reason for other parts of the input, it doesn't even fit the training"}, {"start": 579.68, "end": 583.52, "text": " data well, and so it underfits for part of the input."}, {"start": 583.52, "end": 589.04, "text": " In this example, which looks artificial because it's a single feature input, we fit the"}, {"start": 589.04, "end": 594.1999999999999, "text": " training set really well and we overfit in part of the input, and we don't even fit the"}, {"start": 594.1999999999999, "end": 597.7199999999999, "text": " training data well and we underfit in part of the input."}, {"start": 597.7199999999999, "end": 602.5999999999999, "text": " And that's how in some applications you can unfortunately end up with both high bias and"}, {"start": 602.5999999999999, "end": 605.28, "text": " high variance."}, {"start": 605.28, "end": 609.04, "text": " And the indicator for that will be if the algorithm does poorly on the training set"}, {"start": 609.04, "end": 612.28, "text": " and it even does much worse than on the training set."}, {"start": 612.28, "end": 617.7199999999999, "text": " For most learning applications, you probably have primarily a high bias or a high variance"}, {"start": 617.72, "end": 622.4, "text": " problem rather than both at the same time, but it is possible sometimes that both are"}, {"start": 622.4, "end": 623.88, "text": " the same time."}, {"start": 623.88, "end": 628.0400000000001, "text": " So I know that there's a lot to process, there are a lot of concepts on the slides, but the"}, {"start": 628.0400000000001, "end": 634.48, "text": " key takeaways are high bias means it's not even doing well on the training set, and high"}, {"start": 634.48, "end": 640.9200000000001, "text": " variance means it does much worse on the cross-validation set than the training set."}, {"start": 640.9200000000001, "end": 645.44, "text": " Whenever I'm training a machine learning algorithm, I will almost always try to figure out to"}, {"start": 645.44, "end": 651.4000000000001, "text": " what extent the algorithm has a high bias or underfitting versus a high variance or"}, {"start": 651.4000000000001, "end": 652.8800000000001, "text": " an overfitting problem."}, {"start": 652.8800000000001, "end": 657.2, "text": " And this will give good guidance, as we'll see later this week, on how you can improve"}, {"start": 657.2, "end": 659.48, "text": " the performance of the algorithm."}, {"start": 659.48, "end": 664.48, "text": " But first, let's take a look at how regularization affects the bias and variance of a learning"}, {"start": 664.48, "end": 669.9200000000001, "text": " algorithm, because that will help you better understand when you should use regularization."}, {"start": 669.92, "end": 676.92, "text": " Let's take a look at that in the next video."}]