4.3 Neural Networks Intuition | Demand Prediction --[Machine Learning | Andrew Ng]
 To illustrate how neural networks work, let's start with an example. We'll use an example from demand prediction in which you look at a product and try to predict, will this product be a top seller or not? Let's take a look. In this example, you're selling t-shirts and you would like to know if a particular t-shirt will be a top seller, you know, yes or no. And you have collected data of different t-shirts that were sold at different prices, as well as which ones became a top seller. This type of application is used by retailers today in order to plan better inventory levels as well as marketing campaigns. If you know what's likely to be a top seller, you would plan, for example, to just purchase more of that stock in advance. So in this example, the input feature X is the price of the t-shirt. And so that's the input to the learning algorithm. And if you apply logistic regression to fit a sigmoid function to the data that might look like that, then the output of your prediction might look like this, 1 over 1 plus e to the negative wx plus b. Previously we had written this as f of x as the output of the learning algorithm. In order to set this up to build a neural network, I'm going to switch the terminology a little bit and use the alphabet a to denounce the output of this logistic regression algorithm. The term a stands for activation, and it's actually a term from neuroscience, and it refers to how much a neuron is sending a high output to other neurons downstream from it. It turns out that this logistic regression unit, or this little logistic regression algorithm, can be thought of as a very simplified model of a single neuron in the brain, where what the neuron does is it takes as input the price x, and then it computes this formula on top, and it outputs the number a, which is computed via this formula, and it outputs the probability of this t-shirt being a top seller. Another way to think of a neuron is as a tiny little computer whose only job is to input one number, or a few numbers, such as a price, and then to output one number or maybe a few other numbers, which in this case is the probability of the t-shirt being a top seller. As I alluded in the previous video, a logistic regression algorithm is much simpler than what any biological neuron in your brain or mind does, which is why the artificial neural network is such a vastly oversimplified model of the human brain, even though in practice, as you know, deep learning algorithms do work very well. Given this description of a single neuron, building a neural network now just requires taking a bunch of these neurons and wiring them together or putting them together. Let's now look at a more complex example of demand prediction. In this example, we're going to have four features to predict whether or not a t-shirt is a top seller. The features are the price of the t-shirt, the shipping costs, the amount of marketing of that particular t-shirt, as well as the material quality. Is this a high quality thick cotton, or is this maybe a lower quality material? Now you might suspect that whether or not a t-shirt becomes a top seller actually depends on a few factors. First, what is the affordability of this t-shirt? Second is what's the degree of awareness of this t-shirt that potential buyers have? And third is perceived quality. Do buyers or potential buyers think this is a high quality t-shirt? And so what I'm going to do is create one artificial neuron to try to estimate the probability that this t-shirt is perceived as highly affordable. And affordability is mainly a function of price and shipping costs because the total amount you have to pay is the sum of the price plus the shipping costs. And so we're going to use a little neuron here, a logistic regression unit, to input price and shipping costs and predict, do people think this is affordable? Second, I'm going to create another artificial neuron here to estimate, is there high awareness of this? And awareness in this case is mainly a function of the marketing of the t-shirt. And finally, going to create another neuron to estimate, do people perceive this to be of high quality? And that may mainly be a function of the price of the t-shirt and of the material quality. Price is a factor here because fortunately or unfortunately, if there's a very high price t-shirt, people will sometimes perceive that to be of high quality because if it's very expensive then maybe people think it's got to be of high quality. Given these estimates of affordability, awareness, and perceived quality, we then wire the outputs of these three neurons to another neuron here on the right that then is another logistic regression unit that finally inputs those three numbers and outputs the probability of this t-shirt being a top seller. So in the terminology of neural networks, we're going to group these three neurons together into what's called a layer, and a layer is a grouping of neurons which take as input the same or similar features and that in turn outputs a few numbers together. So these three neurons on the left form one layer, which is why I drew them on top of each other. And the single neuron on the right is also one layer. The layer on the left has three neurons, so a layer can have multiple neurons, or it can also have a single neuron, as in the case of this layer on the right. This layer on the right is also called the output layer because the output of this final neuron is the output probability predicted by the neural network. In the terminology of neural networks, we're also going to call affordability, awareness, and perceived quality to be activations. The term activations comes from biological neurons, and it refers to the degree that a biological neuron is sending a high output value or sending many electrical impulses to other neurons, to the downstream from it. And so these numbers on affordability, awareness, and perceived quality are the activations of these three neurons in this layer. And also, this output probability is the activation of this neuron shown here on the right. So, this particular neural network, therefore, carries out computations as follows. It inputs four numbers, then this layer of the neural network uses those four numbers to compute three new numbers, also called activation values, and then the final layer, the output layer of the neural network, uses those three numbers to compute one number. And in a neural network, this list of four numbers is also called the input layer, and that's just a list of four numbers. Now, there's one simplification I'd like to make to this neural network, which is the way I've described it so far, we had to go through the neurons one at a time and decide what inputs it would take from the previous layer. So for example, we said affordability is a function of just price and shipping costs and awareness is a function of just marketing and so on. But if you're building a large neural network, it'd be a lot of work to go through and manually decide which neurons should take which features as inputs. The way a neural network is implemented in practice, each neuron in a certain layer, say this layer in the middle, will have access to every feature, to every value from the previous layer, from the input layer, which is why I'm now drawing arrows from every input feature to every one of these neurons shown here in the middle. And you can imagine that if you're trying to predict affordability and it knows what the price shipping costs, marketing and material may be, or learn to ignore marketing and material and just figure out through setting the parameters appropriately to only focus on the subset of features that are most relevant to affordability. To further simplify the notation and the description of this neural network, I'm going to take these four input features and write them as a vector x and we're going to view the neural network as having four features that comprise this feature vector x and this feature vector is fed to this layer in the middle, which then computes three activation values, that is these three numbers, and these three activation values in turn becomes another vector, which is fed to this final output layer that finally outputs the probability of this t-shirt being a top seller. So that's all a neural network is. It has a few layers where each layer inputs a vector and outputs another vector of numbers, where for example, this layer in the middle inputs four numbers x and outputs three numbers corresponding to affordability, awareness and perceived quality. To add a little bit more terminology, you've seen that this layer is called the output layer and this layer is called the input layer. To give the layer in the middle a name as well, this layer in the middle is called a hidden layer. I know that this is maybe not the best or the most intuitive name, but that terminology comes from that when you have a training set, in the training set, you get to observe both x and y, your data set tells you what is x and what is y, and so you get data that tells you what are the correct inputs and the correct outputs, but your data set doesn't tell you what are the correct values for affordability, awareness and perceived quality. And so the correct values for those are hidden, you don't see them in the training set, which is why this layer in the middle is called a hidden layer. I'd like to share with you another way of thinking about neural networks that I found useful for building my intuition about it. Just let me cover up the left half of this diagram and see what we're left with. What you see here is that there is a logistic regression algorithm or logistic regression unit that is taking as input affordability, awareness and perceived quality of a t-shirt and using these three features to estimate the probability of the t-shirt being a top seller. So this is just logistic regression. But the cool thing about this is rather than using the original features, price, shipping, cost, marketing and so on, is using a new, maybe better set of features, affordability, awareness and perceived quality that are hopefully more predictive of whether or not this t-shirt will be a top seller. So one way to think of this neural network is just logistic regression, but it is a version of logistic regression that can learn its own features that makes it easier to make accurate predictions. In fact, you might remember from the previous week, this housing example where we said that if you want to predict the price of the house, you might take the frontage or the width of lots and multiply that by the depth of a lot to construct a more complex feature, X1 times X2, which was the size of the lot. So there we were doing manual feature engineering where we had to look at the features X1 and X2 and decide by hand how to combine them together to come up with better features. What the neural network does is instead of you needing to manually engineer the features, it can learn, as you see later, its own features to make the learning problem easier for itself. So this is what makes neural networks one of the most powerful learning algorithms in the world today. So to summarize, a neural network does this. The input layer has a vector of features, four numbers in this example. It is input to the hidden layer, which outputs three numbers. And I'm going to use a vector to denote this vector of activations that this hidden layer outputs. And then the output layer takes this input, those three numbers and outputs one number, which would be the final activation or the final prediction of the neural network. One note, even though I previously described this neural network as computing affordability, awareness, and perceived quality, one of the really nice properties of a neural network is when you train it from data, you don't need to go and explicitly decide what are the features such as affordability and so on that a neural network should compute. Then it will figure out all by itself what are the features it wants to use in this hidden layer. And that's what makes it such a powerful learning algorithm. So you've seen here one example of a neural network, and this neural network has a single layer that is a hidden layer. Let's take a look at some other examples of neural networks, specifically examples with more than one hidden layer. Here's an example. This neural network has an input feature vector X that is fed to one hidden layer, and I'm going to call this the first hidden layer. And so if this hidden layer has three neurons, it will then output a vector of three activation values. These three numbers can then be input to a second hidden layer. And if the second hidden layer has two neurons, two logistic units, then this second hidden layer will output another vector of now two activation values that maybe goes to the output layer that then outputs the neural network's final prediction. Or here's another example. Here's a neural network that has this input go to the first hidden layer, that the output to the first hidden layer goes to the second hidden layer, goes to the third hidden layer, and then finally to the output layer. When you're building your own neural network, one of the decisions you need to make is how many hidden layers do you want and how many neurons do you want each hidden layer to have. And this question of how many hidden layers and how many neurons per hidden layer is a question of the architecture of the neural network. You learn later in this course some tips for choosing an appropriate architecture for a neural network. But choosing the right number of hidden layers and number of hidden units per layer can have an impact on the performance of your learning algorithm as well. So later in this course, you learn how to choose a good architecture for your neural network as well. Oh, and by the way, in some of the literature, you see this type of neural network with multiple layers like this called a multi-layer perceptron. So if you see that, that just refers to a neural network that looks like what you're seeing here on the slide. So that's a neural network. I know we went through a lot of this video. So thank you for sticking with me, but you now know how a neural network works. In the next video, let's take a look at how these ideas can be applied to other applications as well. In particular, we'll take a look at the computer vision application of face recognition. Let's go on to the next video.
[{"start": 0.0, "end": 5.44, "text": " To illustrate how neural networks work, let's start with an example."}, {"start": 5.44, "end": 10.0, "text": " We'll use an example from demand prediction in which you look at a product and try to"}, {"start": 10.0, "end": 13.44, "text": " predict, will this product be a top seller or not?"}, {"start": 13.44, "end": 14.44, "text": " Let's take a look."}, {"start": 14.44, "end": 20.28, "text": " In this example, you're selling t-shirts and you would like to know if a particular"}, {"start": 20.28, "end": 23.64, "text": " t-shirt will be a top seller, you know, yes or no."}, {"start": 23.64, "end": 28.64, "text": " And you have collected data of different t-shirts that were sold at different prices, as well"}, {"start": 28.64, "end": 31.48, "text": " as which ones became a top seller."}, {"start": 31.48, "end": 38.480000000000004, "text": " This type of application is used by retailers today in order to plan better inventory levels"}, {"start": 38.480000000000004, "end": 40.28, "text": " as well as marketing campaigns."}, {"start": 40.28, "end": 44.900000000000006, "text": " If you know what's likely to be a top seller, you would plan, for example, to just purchase"}, {"start": 44.900000000000006, "end": 47.32, "text": " more of that stock in advance."}, {"start": 47.32, "end": 53.68, "text": " So in this example, the input feature X is the price of the t-shirt."}, {"start": 53.68, "end": 56.82, "text": " And so that's the input to the learning algorithm."}, {"start": 56.82, "end": 62.82, "text": " And if you apply logistic regression to fit a sigmoid function to the data that might"}, {"start": 62.82, "end": 68.96000000000001, "text": " look like that, then the output of your prediction might look like this, 1 over 1 plus e to the"}, {"start": 68.96000000000001, "end": 71.84, "text": " negative wx plus b."}, {"start": 71.84, "end": 77.84, "text": " Previously we had written this as f of x as the output of the learning algorithm."}, {"start": 77.84, "end": 81.96000000000001, "text": " In order to set this up to build a neural network, I'm going to switch the terminology"}, {"start": 81.96, "end": 89.16, "text": " a little bit and use the alphabet a to denounce the output of this logistic regression algorithm."}, {"start": 89.16, "end": 94.44, "text": " The term a stands for activation, and it's actually a term from neuroscience, and it"}, {"start": 94.44, "end": 102.83999999999999, "text": " refers to how much a neuron is sending a high output to other neurons downstream from it."}, {"start": 102.83999999999999, "end": 109.0, "text": " It turns out that this logistic regression unit, or this little logistic regression algorithm,"}, {"start": 109.0, "end": 115.48, "text": " can be thought of as a very simplified model of a single neuron in the brain, where what"}, {"start": 115.48, "end": 124.08, "text": " the neuron does is it takes as input the price x, and then it computes this formula on top,"}, {"start": 124.08, "end": 130.48, "text": " and it outputs the number a, which is computed via this formula, and it outputs the probability"}, {"start": 130.48, "end": 133.44, "text": " of this t-shirt being a top seller."}, {"start": 133.44, "end": 141.0, "text": " Another way to think of a neuron is as a tiny little computer whose only job is to input"}, {"start": 141.0, "end": 147.48, "text": " one number, or a few numbers, such as a price, and then to output one number or maybe a few"}, {"start": 147.48, "end": 154.4, "text": " other numbers, which in this case is the probability of the t-shirt being a top seller."}, {"start": 154.4, "end": 160.84, "text": " As I alluded in the previous video, a logistic regression algorithm is much simpler than"}, {"start": 160.84, "end": 166.16, "text": " what any biological neuron in your brain or mind does, which is why the artificial neural"}, {"start": 166.16, "end": 172.24, "text": " network is such a vastly oversimplified model of the human brain, even though in practice,"}, {"start": 172.24, "end": 176.12, "text": " as you know, deep learning algorithms do work very well."}, {"start": 176.12, "end": 181.64000000000001, "text": " Given this description of a single neuron, building a neural network now just requires"}, {"start": 181.64000000000001, "end": 186.96, "text": " taking a bunch of these neurons and wiring them together or putting them together."}, {"start": 186.96, "end": 190.64000000000001, "text": " Let's now look at a more complex example of demand prediction."}, {"start": 190.64, "end": 196.11999999999998, "text": " In this example, we're going to have four features to predict whether or not a t-shirt"}, {"start": 196.11999999999998, "end": 197.2, "text": " is a top seller."}, {"start": 197.2, "end": 202.67999999999998, "text": " The features are the price of the t-shirt, the shipping costs, the amount of marketing"}, {"start": 202.67999999999998, "end": 206.67999999999998, "text": " of that particular t-shirt, as well as the material quality."}, {"start": 206.67999999999998, "end": 212.83999999999997, "text": " Is this a high quality thick cotton, or is this maybe a lower quality material?"}, {"start": 212.83999999999997, "end": 218.51999999999998, "text": " Now you might suspect that whether or not a t-shirt becomes a top seller actually depends"}, {"start": 218.51999999999998, "end": 219.92, "text": " on a few factors."}, {"start": 219.92, "end": 223.95999999999998, "text": " First, what is the affordability of this t-shirt?"}, {"start": 223.95999999999998, "end": 229.6, "text": " Second is what's the degree of awareness of this t-shirt that potential buyers have?"}, {"start": 229.6, "end": 231.88, "text": " And third is perceived quality."}, {"start": 231.88, "end": 236.64, "text": " Do buyers or potential buyers think this is a high quality t-shirt?"}, {"start": 236.64, "end": 244.23999999999998, "text": " And so what I'm going to do is create one artificial neuron to try to estimate the probability"}, {"start": 244.23999999999998, "end": 248.27999999999997, "text": " that this t-shirt is perceived as highly affordable."}, {"start": 248.28, "end": 253.04, "text": " And affordability is mainly a function of price and shipping costs because the total"}, {"start": 253.04, "end": 256.56, "text": " amount you have to pay is the sum of the price plus the shipping costs."}, {"start": 256.56, "end": 260.68, "text": " And so we're going to use a little neuron here, a logistic regression unit, to input"}, {"start": 260.68, "end": 265.32, "text": " price and shipping costs and predict, do people think this is affordable?"}, {"start": 265.32, "end": 271.56, "text": " Second, I'm going to create another artificial neuron here to estimate, is there high awareness"}, {"start": 271.56, "end": 272.62, "text": " of this?"}, {"start": 272.62, "end": 278.16, "text": " And awareness in this case is mainly a function of the marketing of the t-shirt."}, {"start": 278.16, "end": 284.04, "text": " And finally, going to create another neuron to estimate, do people perceive this to be"}, {"start": 284.04, "end": 286.12, "text": " of high quality?"}, {"start": 286.12, "end": 292.68, "text": " And that may mainly be a function of the price of the t-shirt and of the material quality."}, {"start": 292.68, "end": 297.8, "text": " Price is a factor here because fortunately or unfortunately, if there's a very high price"}, {"start": 297.8, "end": 302.92, "text": " t-shirt, people will sometimes perceive that to be of high quality because if it's very"}, {"start": 302.92, "end": 307.32000000000005, "text": " expensive then maybe people think it's got to be of high quality."}, {"start": 307.32, "end": 313.44, "text": " Given these estimates of affordability, awareness, and perceived quality, we then wire the outputs"}, {"start": 313.44, "end": 318.84, "text": " of these three neurons to another neuron here on the right that then is another logistic"}, {"start": 318.84, "end": 324.62, "text": " regression unit that finally inputs those three numbers and outputs the probability"}, {"start": 324.62, "end": 327.46, "text": " of this t-shirt being a top seller."}, {"start": 327.46, "end": 333.1, "text": " So in the terminology of neural networks, we're going to group these three neurons"}, {"start": 333.1, "end": 340.52000000000004, "text": " together into what's called a layer, and a layer is a grouping of neurons which take"}, {"start": 340.52000000000004, "end": 347.64000000000004, "text": " as input the same or similar features and that in turn outputs a few numbers together."}, {"start": 347.64000000000004, "end": 352.12, "text": " So these three neurons on the left form one layer, which is why I drew them on top of"}, {"start": 352.12, "end": 353.40000000000003, "text": " each other."}, {"start": 353.40000000000003, "end": 358.04, "text": " And the single neuron on the right is also one layer."}, {"start": 358.04, "end": 362.8, "text": " The layer on the left has three neurons, so a layer can have multiple neurons, or it can"}, {"start": 362.8, "end": 367.68, "text": " also have a single neuron, as in the case of this layer on the right."}, {"start": 367.68, "end": 374.0, "text": " This layer on the right is also called the output layer because the output of this final"}, {"start": 374.0, "end": 379.32, "text": " neuron is the output probability predicted by the neural network."}, {"start": 379.32, "end": 384.8, "text": " In the terminology of neural networks, we're also going to call affordability, awareness,"}, {"start": 384.8, "end": 388.32, "text": " and perceived quality to be activations."}, {"start": 388.32, "end": 393.24, "text": " The term activations comes from biological neurons, and it refers to the degree that"}, {"start": 393.24, "end": 398.4, "text": " a biological neuron is sending a high output value or sending many electrical impulses"}, {"start": 398.4, "end": 401.36, "text": " to other neurons, to the downstream from it."}, {"start": 401.36, "end": 406.71999999999997, "text": " And so these numbers on affordability, awareness, and perceived quality are the activations"}, {"start": 406.71999999999997, "end": 410.12, "text": " of these three neurons in this layer."}, {"start": 410.12, "end": 418.03999999999996, "text": " And also, this output probability is the activation of this neuron shown here on the right."}, {"start": 418.04, "end": 423.88, "text": " So, this particular neural network, therefore, carries out computations as follows."}, {"start": 423.88, "end": 429.24, "text": " It inputs four numbers, then this layer of the neural network uses those four numbers"}, {"start": 429.24, "end": 435.64000000000004, "text": " to compute three new numbers, also called activation values, and then the final layer,"}, {"start": 435.64000000000004, "end": 442.24, "text": " the output layer of the neural network, uses those three numbers to compute one number."}, {"start": 442.24, "end": 450.84000000000003, "text": " And in a neural network, this list of four numbers is also called the input layer, and"}, {"start": 450.84000000000003, "end": 453.04, "text": " that's just a list of four numbers."}, {"start": 453.04, "end": 459.92, "text": " Now, there's one simplification I'd like to make to this neural network, which is the"}, {"start": 459.92, "end": 465.08, "text": " way I've described it so far, we had to go through the neurons one at a time and decide"}, {"start": 465.08, "end": 468.40000000000003, "text": " what inputs it would take from the previous layer."}, {"start": 468.4, "end": 473.08, "text": " So for example, we said affordability is a function of just price and shipping costs"}, {"start": 473.08, "end": 477.0, "text": " and awareness is a function of just marketing and so on."}, {"start": 477.0, "end": 481.08, "text": " But if you're building a large neural network, it'd be a lot of work to go through and manually"}, {"start": 481.08, "end": 485.91999999999996, "text": " decide which neurons should take which features as inputs."}, {"start": 485.91999999999996, "end": 491.32, "text": " The way a neural network is implemented in practice, each neuron in a certain layer,"}, {"start": 491.32, "end": 498.15999999999997, "text": " say this layer in the middle, will have access to every feature, to every value from the"}, {"start": 498.16, "end": 505.16, "text": " previous layer, from the input layer, which is why I'm now drawing arrows from every input"}, {"start": 505.16, "end": 510.18, "text": " feature to every one of these neurons shown here in the middle."}, {"start": 510.18, "end": 515.28, "text": " And you can imagine that if you're trying to predict affordability and it knows what"}, {"start": 515.28, "end": 521.0, "text": " the price shipping costs, marketing and material may be, or learn to ignore marketing and material"}, {"start": 521.0, "end": 526.84, "text": " and just figure out through setting the parameters appropriately to only focus on the subset"}, {"start": 526.84, "end": 530.9200000000001, "text": " of features that are most relevant to affordability."}, {"start": 530.9200000000001, "end": 537.08, "text": " To further simplify the notation and the description of this neural network, I'm going to take"}, {"start": 537.08, "end": 544.08, "text": " these four input features and write them as a vector x and we're going to view the neural"}, {"start": 544.08, "end": 552.6800000000001, "text": " network as having four features that comprise this feature vector x and this feature vector"}, {"start": 552.68, "end": 559.52, "text": " is fed to this layer in the middle, which then computes three activation values, that"}, {"start": 559.52, "end": 567.9599999999999, "text": " is these three numbers, and these three activation values in turn becomes another vector, which"}, {"start": 567.9599999999999, "end": 576.1999999999999, "text": " is fed to this final output layer that finally outputs the probability of this t-shirt being"}, {"start": 576.1999999999999, "end": 578.04, "text": " a top seller."}, {"start": 578.04, "end": 580.3399999999999, "text": " So that's all a neural network is."}, {"start": 580.34, "end": 588.5600000000001, "text": " It has a few layers where each layer inputs a vector and outputs another vector of numbers,"}, {"start": 588.5600000000001, "end": 595.14, "text": " where for example, this layer in the middle inputs four numbers x and outputs three numbers"}, {"start": 595.14, "end": 598.88, "text": " corresponding to affordability, awareness and perceived quality."}, {"start": 598.88, "end": 605.6800000000001, "text": " To add a little bit more terminology, you've seen that this layer is called the output"}, {"start": 605.68, "end": 610.0799999999999, "text": " layer and this layer is called the input layer."}, {"start": 610.0799999999999, "end": 614.64, "text": " To give the layer in the middle a name as well, this layer in the middle is called a"}, {"start": 614.64, "end": 616.76, "text": " hidden layer."}, {"start": 616.76, "end": 622.2399999999999, "text": " I know that this is maybe not the best or the most intuitive name, but that terminology"}, {"start": 622.2399999999999, "end": 628.4799999999999, "text": " comes from that when you have a training set, in the training set, you get to observe both"}, {"start": 628.4799999999999, "end": 634.92, "text": " x and y, your data set tells you what is x and what is y, and so you get data that tells"}, {"start": 634.92, "end": 640.56, "text": " you what are the correct inputs and the correct outputs, but your data set doesn't tell you"}, {"start": 640.56, "end": 645.3199999999999, "text": " what are the correct values for affordability, awareness and perceived quality."}, {"start": 645.3199999999999, "end": 649.24, "text": " And so the correct values for those are hidden, you don't see them in the training set, which"}, {"start": 649.24, "end": 653.12, "text": " is why this layer in the middle is called a hidden layer."}, {"start": 653.12, "end": 658.28, "text": " I'd like to share with you another way of thinking about neural networks that I found"}, {"start": 658.28, "end": 661.3199999999999, "text": " useful for building my intuition about it."}, {"start": 661.32, "end": 666.36, "text": " Just let me cover up the left half of this diagram and see what we're left with."}, {"start": 666.36, "end": 671.7600000000001, "text": " What you see here is that there is a logistic regression algorithm or logistic regression"}, {"start": 671.7600000000001, "end": 678.1600000000001, "text": " unit that is taking as input affordability, awareness and perceived quality of a t-shirt"}, {"start": 678.1600000000001, "end": 683.6, "text": " and using these three features to estimate the probability of the t-shirt being a top"}, {"start": 683.6, "end": 684.6800000000001, "text": " seller."}, {"start": 684.6800000000001, "end": 688.48, "text": " So this is just logistic regression."}, {"start": 688.48, "end": 694.12, "text": " But the cool thing about this is rather than using the original features, price, shipping,"}, {"start": 694.12, "end": 699.08, "text": " cost, marketing and so on, is using a new, maybe better set of features, affordability,"}, {"start": 699.08, "end": 704.9200000000001, "text": " awareness and perceived quality that are hopefully more predictive of whether or not this t-shirt"}, {"start": 704.9200000000001, "end": 706.52, "text": " will be a top seller."}, {"start": 706.52, "end": 713.8000000000001, "text": " So one way to think of this neural network is just logistic regression, but it is a version"}, {"start": 713.8, "end": 719.16, "text": " of logistic regression that can learn its own features that makes it easier to make"}, {"start": 719.16, "end": 721.04, "text": " accurate predictions."}, {"start": 721.04, "end": 726.76, "text": " In fact, you might remember from the previous week, this housing example where we said that"}, {"start": 726.76, "end": 731.28, "text": " if you want to predict the price of the house, you might take the frontage or the width of"}, {"start": 731.28, "end": 737.4, "text": " lots and multiply that by the depth of a lot to construct a more complex feature, X1 times"}, {"start": 737.4, "end": 740.28, "text": " X2, which was the size of the lot."}, {"start": 740.28, "end": 745.6, "text": " So there we were doing manual feature engineering where we had to look at the features X1 and"}, {"start": 745.6, "end": 751.04, "text": " X2 and decide by hand how to combine them together to come up with better features."}, {"start": 751.04, "end": 756.56, "text": " What the neural network does is instead of you needing to manually engineer the features,"}, {"start": 756.56, "end": 764.24, "text": " it can learn, as you see later, its own features to make the learning problem easier for itself."}, {"start": 764.24, "end": 768.52, "text": " So this is what makes neural networks one of the most powerful learning algorithms in"}, {"start": 768.52, "end": 769.8399999999999, "text": " the world today."}, {"start": 769.84, "end": 773.72, "text": " So to summarize, a neural network does this."}, {"start": 773.72, "end": 778.36, "text": " The input layer has a vector of features, four numbers in this example."}, {"start": 778.36, "end": 783.32, "text": " It is input to the hidden layer, which outputs three numbers."}, {"start": 783.32, "end": 790.52, "text": " And I'm going to use a vector to denote this vector of activations that this hidden layer"}, {"start": 790.52, "end": 791.8000000000001, "text": " outputs."}, {"start": 791.8000000000001, "end": 798.08, "text": " And then the output layer takes this input, those three numbers and outputs one number,"}, {"start": 798.08, "end": 804.44, "text": " which would be the final activation or the final prediction of the neural network."}, {"start": 804.44, "end": 809.12, "text": " One note, even though I previously described this neural network as computing affordability,"}, {"start": 809.12, "end": 813.5600000000001, "text": " awareness, and perceived quality, one of the really nice properties of a neural network"}, {"start": 813.5600000000001, "end": 819.24, "text": " is when you train it from data, you don't need to go and explicitly decide what are"}, {"start": 819.24, "end": 824.32, "text": " the features such as affordability and so on that a neural network should compute."}, {"start": 824.32, "end": 829.72, "text": " Then it will figure out all by itself what are the features it wants to use in this hidden"}, {"start": 829.72, "end": 830.72, "text": " layer."}, {"start": 830.72, "end": 834.08, "text": " And that's what makes it such a powerful learning algorithm."}, {"start": 834.08, "end": 839.48, "text": " So you've seen here one example of a neural network, and this neural network has a single"}, {"start": 839.48, "end": 841.88, "text": " layer that is a hidden layer."}, {"start": 841.88, "end": 846.7600000000001, "text": " Let's take a look at some other examples of neural networks, specifically examples with"}, {"start": 846.7600000000001, "end": 849.6, "text": " more than one hidden layer."}, {"start": 849.6, "end": 851.2800000000001, "text": " Here's an example."}, {"start": 851.28, "end": 858.92, "text": " This neural network has an input feature vector X that is fed to one hidden layer, and I'm"}, {"start": 858.92, "end": 861.42, "text": " going to call this the first hidden layer."}, {"start": 861.42, "end": 868.36, "text": " And so if this hidden layer has three neurons, it will then output a vector of three activation"}, {"start": 868.36, "end": 869.88, "text": " values."}, {"start": 869.88, "end": 875.12, "text": " These three numbers can then be input to a second hidden layer."}, {"start": 875.12, "end": 881.04, "text": " And if the second hidden layer has two neurons, two logistic units, then this second hidden"}, {"start": 881.04, "end": 887.36, "text": " layer will output another vector of now two activation values that maybe goes to the output"}, {"start": 887.36, "end": 891.68, "text": " layer that then outputs the neural network's final prediction."}, {"start": 891.68, "end": 893.48, "text": " Or here's another example."}, {"start": 893.48, "end": 899.0799999999999, "text": " Here's a neural network that has this input go to the first hidden layer, that the output"}, {"start": 899.0799999999999, "end": 902.92, "text": " to the first hidden layer goes to the second hidden layer, goes to the third hidden layer,"}, {"start": 902.92, "end": 906.0999999999999, "text": " and then finally to the output layer."}, {"start": 906.0999999999999, "end": 910.4399999999999, "text": " When you're building your own neural network, one of the decisions you need to make is how"}, {"start": 910.44, "end": 916.5600000000001, "text": " many hidden layers do you want and how many neurons do you want each hidden layer to have."}, {"start": 916.5600000000001, "end": 922.4000000000001, "text": " And this question of how many hidden layers and how many neurons per hidden layer is a"}, {"start": 922.4000000000001, "end": 925.8800000000001, "text": " question of the architecture of the neural network."}, {"start": 925.8800000000001, "end": 931.0, "text": " You learn later in this course some tips for choosing an appropriate architecture for a"}, {"start": 931.0, "end": 932.0, "text": " neural network."}, {"start": 932.0, "end": 937.5200000000001, "text": " But choosing the right number of hidden layers and number of hidden units per layer can have"}, {"start": 937.52, "end": 940.8, "text": " an impact on the performance of your learning algorithm as well."}, {"start": 940.8, "end": 945.68, "text": " So later in this course, you learn how to choose a good architecture for your neural"}, {"start": 945.68, "end": 946.68, "text": " network as well."}, {"start": 946.68, "end": 951.4, "text": " Oh, and by the way, in some of the literature, you see this type of neural network with multiple"}, {"start": 951.4, "end": 954.48, "text": " layers like this called a multi-layer perceptron."}, {"start": 954.48, "end": 958.36, "text": " So if you see that, that just refers to a neural network that looks like what you're"}, {"start": 958.36, "end": 960.86, "text": " seeing here on the slide."}, {"start": 960.86, "end": 963.36, "text": " So that's a neural network."}, {"start": 963.36, "end": 965.52, "text": " I know we went through a lot of this video."}, {"start": 965.52, "end": 970.52, "text": " So thank you for sticking with me, but you now know how a neural network works."}, {"start": 970.52, "end": 975.0, "text": " In the next video, let's take a look at how these ideas can be applied to other applications"}, {"start": 975.0, "end": 976.0, "text": " as well."}, {"start": 976.0, "end": 981.04, "text": " In particular, we'll take a look at the computer vision application of face recognition."}, {"start": 981.04, "end": 996.68, "text": " Let's go on to the next video."}]