8.12 Anomaly Detection | Anomaly detection vs.  supervised learning -- [Machine Learning| Andrew Ng]
 When you have a few positive examples with y equals 1 and a large number of negative examples say y equals 0, when should you use anomaly detection and when should you use supervised learning? The decision is actually quite subtle in some applications, so let me share with you some thoughts and some suggestions for how to pick between these two types of algorithms. An anomaly detection algorithm will typically be the more appropriate choice when you have a very small number of positive examples, 0 to 20 positive examples is not uncommon and a relatively large number of negative examples with which to try to build a model for p of x. Where you recall that the parameters for p of x are learned only from the negative examples and this much smaller set of positive examples is only used in your cross-validation set and test set for parameter tuning and for evaluation. In contrast, if you have a larger number of positive and negative examples, then supervised learning might be more applicable. Now, even if you have only 20 positive training examples, it might be okay to apply a supervised learning algorithm, but it turns out that the way anomaly detection looks at the data set versus the way supervised learning looks at the data set are quite different. Here's the main difference, which is that if you think there are many different types of anomalies or many different types of positive examples, then anomaly detection might be more appropriate. When there are many different ways for an aircraft engine to go wrong and if tomorrow there may be a brand new way for an aircraft engine to have something wrong with it, then your 20, say, positive examples may not cover all of the ways that an aircraft engine could go wrong. That makes it hard for any algorithm to learn from the smallest of the positive examples what the anomalies, what the positive examples look like. Future anomalies may look nothing like any of the anomalous examples we've seen so far. If you believe this to be true for your problem, then I would gravitate to using an anomaly detection algorithm. Because what anomaly detection does is it looks at the normal examples, that is the y equals zero and negative examples, and just try to model what they look like. Something that deviates a lot from normal, it flags as an anomaly, including if there's a brand new way for an aircraft engine to fail that had never been seen before in your data set. In contrast, supervised learning has a different way of looking at the problem. When you apply supervised learning, ideally you would hope to have enough positive examples for the algorithm to get a sense of what the positive examples are like. And with supervised learning, we tend to assume that the future positive examples are likely to be similar to the ones in the training set. So let me illustrate this with one example. If you are using a system to find, say, financial fraud, there are many different ways, unfortunately, that some individuals are trying to commit financial fraud. And unfortunately, there are new types of financial fraud attempts every few months or every year. And what that means is that because they keep on popping up completely new and unique forms of financial fraud, anomaly detection is often used to just look for anything that's different than transactions we've seen in the past. In contrast, if you look at the problem of email spam detection, well, there are many different types of spam email, but even over many years, spam emails keep on trying to sell similar things or get you to go to similar websites and so on. Spam email that you will get in the next few days is much more likely to be similar to spam emails that you have seen in the past. So that's why supervised learning works well for spam because it's trying to detect more of the types of spam emails that you have probably seen in the past in your training set. Whereas if you're trying to detect brand new types of fraud that have never been seen before, then anomaly detection may be more applicable. Let's go through a few more examples. We have already seen fraud detection being one use case of anomaly detection. Although supervised learning is used to define previously observed forms of fraud, and we've seen emails spam classification typically being addressed using supervised learning. You've also seen the example of manufacturing where you may want to find new previously unseen defects, such as if there are brand new ways for an aircraft engine to fail in the future that you still want to detect, even if you don't have any positive example like that in your training set. It turns out that in manufacturing, supervised learning is also used to find defects, more for finding known and previously seen defects. For example, if you are a smartphone maker, you're making cell phones and you know that occasionally your machine for making the case of the smartphone will accidentally scratch the cover. So scratches are a common defect on smartphones. And so you can get enough training examples of scratched smartphones corresponding to a label Y equals one and just train the system to decide if a new smartphone that you just manufactured has any scratches in it. And the difference is if you just see scratched smartphones over and over, and you want to check if your phones are scratched, then supervised learning works well. Whereas if you suspect that there are going to be brand new ways for something to go wrong in the future, then anomaly detection will work well. Some other examples, you've heard me talk about monitoring machines in the data center, especially if a machine has been hacked, it can behave differently in a brand new way, unlike any previous way it has behaved. So that would feel more like an anomaly detection application. In fact, one theme is that many security related applications because hackers are often finding brand new ways to hack into systems, many security related applications will use anomaly detection. Whereas returning to supervised learning, if you want to learn to predict the weather, well, there's only a handful types of weather that you typically see, is it sunny, rainy, is it going to snow? And so because you see the same output labels over and over, weather prediction would tend to be a supervised learning task. Or if you want to use the symptoms of a patient to see if the patient has a specific disease that you've seen before, then that would also tend to be a supervised learning application. So I hope that gives you a framework for deciding when you have a small set of positive examples, as well as maybe a large set of negative examples, whether to use anomaly detection or supervised learning. Anomaly detection tries to find brand new positive examples that may be unlike anything you've seen before. Whereas supervised learning looks at your positive examples and tries to decide if a future example is similar to the positive examples that you've already seen. Now, it turns out that when building an anomaly detection algorithm, the choice of features is very important. And when building anomaly detection systems, I often spend a bit of time trying to tune the features I use for the system. In the next video, let me share some practical tips on how to tune the features you feed to anomaly detection algorithm.
[{"start": 0.0, "end": 8.44, "text": " When you have a few positive examples with y equals 1 and a large number of negative"}, {"start": 8.44, "end": 13.86, "text": " examples say y equals 0, when should you use anomaly detection and when should you use"}, {"start": 13.86, "end": 19.5, "text": " supervised learning? The decision is actually quite subtle in some applications, so let"}, {"start": 19.5, "end": 24.64, "text": " me share with you some thoughts and some suggestions for how to pick between these two types of"}, {"start": 24.64, "end": 30.8, "text": " algorithms. An anomaly detection algorithm will typically be the more appropriate choice"}, {"start": 30.8, "end": 38.92, "text": " when you have a very small number of positive examples, 0 to 20 positive examples is not"}, {"start": 38.92, "end": 46.58, "text": " uncommon and a relatively large number of negative examples with which to try to build"}, {"start": 46.58, "end": 53.38, "text": " a model for p of x. Where you recall that the parameters for p of x are learned only"}, {"start": 53.38, "end": 59.32, "text": " from the negative examples and this much smaller set of positive examples is only used in your"}, {"start": 59.32, "end": 65.2, "text": " cross-validation set and test set for parameter tuning and for evaluation. In contrast, if"}, {"start": 65.2, "end": 71.76, "text": " you have a larger number of positive and negative examples, then supervised learning might be"}, {"start": 71.76, "end": 79.96000000000001, "text": " more applicable. Now, even if you have only 20 positive training examples, it might be"}, {"start": 79.96, "end": 86.8, "text": " okay to apply a supervised learning algorithm, but it turns out that the way anomaly detection"}, {"start": 86.8, "end": 92.72, "text": " looks at the data set versus the way supervised learning looks at the data set are quite different."}, {"start": 92.72, "end": 98.67999999999999, "text": " Here's the main difference, which is that if you think there are many different types"}, {"start": 98.67999999999999, "end": 105.66, "text": " of anomalies or many different types of positive examples, then anomaly detection might be"}, {"start": 105.66, "end": 111.6, "text": " more appropriate. When there are many different ways for an aircraft engine to go wrong and"}, {"start": 111.6, "end": 117.08, "text": " if tomorrow there may be a brand new way for an aircraft engine to have something wrong"}, {"start": 117.08, "end": 124.88, "text": " with it, then your 20, say, positive examples may not cover all of the ways that an aircraft"}, {"start": 124.88, "end": 130.0, "text": " engine could go wrong. That makes it hard for any algorithm to learn from the smallest"}, {"start": 130.0, "end": 136.12, "text": " of the positive examples what the anomalies, what the positive examples look like. Future"}, {"start": 136.12, "end": 141.64, "text": " anomalies may look nothing like any of the anomalous examples we've seen so far. If you"}, {"start": 141.64, "end": 147.28, "text": " believe this to be true for your problem, then I would gravitate to using an anomaly"}, {"start": 147.28, "end": 153.52, "text": " detection algorithm. Because what anomaly detection does is it looks at the normal examples,"}, {"start": 153.52, "end": 159.64, "text": " that is the y equals zero and negative examples, and just try to model what they look like."}, {"start": 159.64, "end": 164.39999999999998, "text": " Something that deviates a lot from normal, it flags as an anomaly, including if there's"}, {"start": 164.39999999999998, "end": 170.16, "text": " a brand new way for an aircraft engine to fail that had never been seen before in your"}, {"start": 170.16, "end": 175.51999999999998, "text": " data set. In contrast, supervised learning has a different way of looking at the problem."}, {"start": 175.51999999999998, "end": 180.76, "text": " When you apply supervised learning, ideally you would hope to have enough positive examples"}, {"start": 180.76, "end": 185.72, "text": " for the algorithm to get a sense of what the positive examples are like. And with supervised"}, {"start": 185.72, "end": 191.72, "text": " learning, we tend to assume that the future positive examples are likely to be similar"}, {"start": 191.72, "end": 198.72, "text": " to the ones in the training set. So let me illustrate this with one example. If you are"}, {"start": 198.72, "end": 205.28, "text": " using a system to find, say, financial fraud, there are many different ways, unfortunately,"}, {"start": 205.28, "end": 211.28, "text": " that some individuals are trying to commit financial fraud. And unfortunately, there"}, {"start": 211.28, "end": 216.96, "text": " are new types of financial fraud attempts every few months or every year. And what that"}, {"start": 216.96, "end": 223.64, "text": " means is that because they keep on popping up completely new and unique forms of financial"}, {"start": 223.64, "end": 229.92000000000002, "text": " fraud, anomaly detection is often used to just look for anything that's different than"}, {"start": 229.92000000000002, "end": 236.88, "text": " transactions we've seen in the past. In contrast, if you look at the problem of email spam detection,"}, {"start": 236.88, "end": 243.4, "text": " well, there are many different types of spam email, but even over many years, spam emails"}, {"start": 243.4, "end": 250.44, "text": " keep on trying to sell similar things or get you to go to similar websites and so on. Spam"}, {"start": 250.44, "end": 255.72, "text": " email that you will get in the next few days is much more likely to be similar to spam"}, {"start": 255.72, "end": 262.4, "text": " emails that you have seen in the past. So that's why supervised learning works well"}, {"start": 262.4, "end": 267.91999999999996, "text": " for spam because it's trying to detect more of the types of spam emails that you have"}, {"start": 267.91999999999996, "end": 273.12, "text": " probably seen in the past in your training set. Whereas if you're trying to detect brand"}, {"start": 273.12, "end": 278.08, "text": " new types of fraud that have never been seen before, then anomaly detection may be more"}, {"start": 278.08, "end": 286.21999999999997, "text": " applicable. Let's go through a few more examples. We have already seen fraud detection being"}, {"start": 286.21999999999997, "end": 292.14, "text": " one use case of anomaly detection. Although supervised learning is used to define previously"}, {"start": 292.14, "end": 298.38, "text": " observed forms of fraud, and we've seen emails spam classification typically being addressed"}, {"start": 298.38, "end": 305.2, "text": " using supervised learning. You've also seen the example of manufacturing where you may"}, {"start": 305.2, "end": 312.0, "text": " want to find new previously unseen defects, such as if there are brand new ways for an"}, {"start": 312.0, "end": 316.0, "text": " aircraft engine to fail in the future that you still want to detect, even if you don't"}, {"start": 316.0, "end": 322.76, "text": " have any positive example like that in your training set. It turns out that in manufacturing,"}, {"start": 322.76, "end": 328.24, "text": " supervised learning is also used to find defects, more for finding known and previously seen"}, {"start": 328.24, "end": 334.64, "text": " defects. For example, if you are a smartphone maker, you're making cell phones and you know"}, {"start": 334.64, "end": 340.28, "text": " that occasionally your machine for making the case of the smartphone will accidentally"}, {"start": 340.28, "end": 347.2, "text": " scratch the cover. So scratches are a common defect on smartphones. And so you can get"}, {"start": 347.2, "end": 354.0, "text": " enough training examples of scratched smartphones corresponding to a label Y equals one and"}, {"start": 354.0, "end": 359.67999999999995, "text": " just train the system to decide if a new smartphone that you just manufactured has any scratches"}, {"start": 359.67999999999995, "end": 364.84, "text": " in it. And the difference is if you just see scratched smartphones over and over, and you"}, {"start": 364.84, "end": 370.2, "text": " want to check if your phones are scratched, then supervised learning works well. Whereas"}, {"start": 370.2, "end": 373.96, "text": " if you suspect that there are going to be brand new ways for something to go wrong in"}, {"start": 373.96, "end": 379.35999999999996, "text": " the future, then anomaly detection will work well. Some other examples, you've heard me"}, {"start": 379.35999999999996, "end": 384.88, "text": " talk about monitoring machines in the data center, especially if a machine has been hacked,"}, {"start": 384.88, "end": 389.84, "text": " it can behave differently in a brand new way, unlike any previous way it has behaved. So"}, {"start": 389.84, "end": 396.15999999999997, "text": " that would feel more like an anomaly detection application. In fact, one theme is that many"}, {"start": 396.15999999999997, "end": 402.64, "text": " security related applications because hackers are often finding brand new ways to hack into"}, {"start": 402.64, "end": 407.76, "text": " systems, many security related applications will use anomaly detection. Whereas returning"}, {"start": 407.76, "end": 413.4, "text": " to supervised learning, if you want to learn to predict the weather, well, there's only"}, {"start": 413.4, "end": 419.71999999999997, "text": " a handful types of weather that you typically see, is it sunny, rainy, is it going to snow?"}, {"start": 419.72, "end": 425.04, "text": " And so because you see the same output labels over and over, weather prediction would tend"}, {"start": 425.04, "end": 430.40000000000003, "text": " to be a supervised learning task. Or if you want to use the symptoms of a patient to see"}, {"start": 430.40000000000003, "end": 434.96000000000004, "text": " if the patient has a specific disease that you've seen before, then that would also tend"}, {"start": 434.96000000000004, "end": 441.0, "text": " to be a supervised learning application. So I hope that gives you a framework for deciding"}, {"start": 441.0, "end": 446.6, "text": " when you have a small set of positive examples, as well as maybe a large set of negative examples,"}, {"start": 446.6, "end": 452.40000000000003, "text": " whether to use anomaly detection or supervised learning. Anomaly detection tries to find"}, {"start": 452.40000000000003, "end": 457.8, "text": " brand new positive examples that may be unlike anything you've seen before. Whereas supervised"}, {"start": 457.8, "end": 463.16, "text": " learning looks at your positive examples and tries to decide if a future example is similar"}, {"start": 463.16, "end": 469.6, "text": " to the positive examples that you've already seen. Now, it turns out that when building"}, {"start": 469.6, "end": 475.52000000000004, "text": " an anomaly detection algorithm, the choice of features is very important. And when building"}, {"start": 475.52, "end": 480.15999999999997, "text": " anomaly detection systems, I often spend a bit of time trying to tune the features I"}, {"start": 480.15999999999997, "end": 485.4, "text": " use for the system. In the next video, let me share some practical tips on how to tune"}, {"start": 485.4, "end": 506.4, "text": " the features you feed to anomaly detection algorithm."}]