4.5 Neural Networks Model | Neural Network Layer --[Machine Learning | Andrew Ng]
 The fundamental building block of most modern neural networks is a layer of neurons. In this video, you learn how to construct a layer of neurons, and once you have that down, you'll be able to take those building blocks and put them together to form a large neural network. Let's take a look at how a layer of neurons works. Here's the example we had from the demand prediction example where we had four input features that were fed to this layer of three neurons in this hidden layer that then sends this output to this output layer with just one neuron. Let's zoom in to the hidden layer to look at its computations. This hidden layer inputs four numbers, and these four numbers are inputs to each of three neurons, and each of these three neurons is just implementing a little logistic regression unit or a little logistic regression function. Take this first neuron. It has two parameters, W and B, and in fact, to denote that this is the first hidden unit, I'm going to subscript this as W1B1, and what it does is it'll output some activation value A, which is G of W1 in a product of X plus B1, where this is the familiar Z value that you had learned about in logistic regression in the previous course, and G of Z is the familiar logistic function, one over one plus E to the negative Z, and so maybe this ends up being a number 0.3, and that's the activation value A of the first neuron. To denote that this is the first neuron, I'm also going to add a subscript A1 over here, and so A1 may be a number like 0.3. That's a 0.3 chance of this being highly affordable based on the input features. Now let's look at the second neuron. The second neuron has parameters W2 and B2, and this WB or W2B2 are the parameters of the second logistic unit. So it computes A2 equals the logistic function G applied to W2 dot product X plus B2, and this may be some other number, say 0.7, because in this example there's a 0.7 chance that we think the potential buyers will be aware of this t-shirt. And similarly, the third neuron has a third set of parameters, W3B3, and similarly computes an activation value A3 equals G of W3 dot product X plus B3, and that may be, say, 0.2. So in this example, these three neurons output 0.3, 0.7, and 0.2, and this vector of three numbers becomes the vector of activation values A that is then passed to the final output layer of this neural network. Now when you build neural networks with multiple layers, it will be useful to give the layers different numbers. So by convention, this layer is called layer one of the neural network, and this layer is called layer two of the neural network, and the input layer is also sometimes called layer zero, and today there are neural networks that can have dozens or even hundreds of layers. But in order to introduce notation to help us distinguish between the different layers, I'm going to use superscript square bracket one to index into different layers. So in particular, A superscript in square brackets one, I'm going to use as a notation to denote the output of layer one of this hidden layer of this neural network, and similarly, W1B1 here are the parameters of the first unit in layer one of the neural network, so I'm also going to add a superscript in square brackets one here, and W2B2 are the parameters of the second hidden unit or the second hidden neuron in layer one, and so those parameters are also denoted here, W superscript square bracket one, like so. And similarly, I can add superscript square brackets like so to denote that these are the activation values of the hidden units of layer one of this neural network. I know maybe this notation is getting a little bit cluttered, but the thing to remember is whenever you see this superscript square bracket one, that just refers to a quantity that is associated with layer one of the neural network, and if you see superscript square bracket two, that refers to a quantity associated with layer two of the neural network, and similarly for other layers as well, including layer three, layer four, and so on for neural networks with more layers. So that's the computation of layer one of this neural network. This output is this activation vector, A superscript square bracket one, and I'm going to copy this over here because this output, A1, becomes the input to layer two. So now let's zoom into the computation of layer two of this neural network, which is also the output layer. So the input to layer two is the output of layer one. So A1 is this vector 0.3, 0.7, 0.2 that we just computed on the previous part of the slide, and so because the output layer has just a single neuron, all it does is it computes a subscript one that is the output of its first and only neuron as G, the sigmoid function, applied to W subscript one in a product with A superscript square bracket one, so this is the input into this layer, and then plus B1. Here this is the quantity Z that you're familiar with, and G as before is the sigmoid function that you apply to this, and if this results in a number, say 0.84, then that becomes the output of this output layer of the neural network, and in this example, because the output layer has just a single neuron, this output is just a scalar, it's a single number rather than a vector of numbers. Sticking with our notational convention from before, we're going to use A superscript in square brackets two to denote the quantities associated with layer two of this neural network, so A superscript square bracket two is the output of this layer, and so I'm going to also copy this here as the final output of the neural network, and to make the notation consistent, you can also add these superscript square bracket twos to denote that these are the parameters and activation values associated with layer two of the neural network. Once the neural network has computed A2, there's one final optional step that you can choose to implement or not, which is if you want a binary prediction, one or zero, is this a top seller, yes or no, is you can take the number A superscript square brackets two, subscript one, and this is the number 0.84 that we computed, and threshold is at 0.5, so if it's greater than 0.5, you can predict y hat equals one, and if it's less than 0.5, then predict y hat equals zero, and we saw this thresholding as well when we learned about logistic regression in the first course of the specialization. So if you wish, this then gives you the final prediction y hat as either one or zero if you don't want just a probability of it being a top seller. So that's how a neural network works. Every layer inputs a vector of numbers and applies a bunch of logistic regression units to it and then computes another vector of numbers that then gets passed from layer to layer until you get the final output layers computation, which is a prediction of the neural network that you can either threshold at 0.5 or not to come up with the final prediction, and with that, let's go on to use this foundation we've built now to look at some even more complex, even larger neural network models, and I hope that by seeing more examples, this concept of layers and how to put them together to build a neural network will become even clearer. So let's go on to the next video.
[{"start": 0.0, "end": 8.68, "text": " The fundamental building block of most modern neural networks is a layer of neurons."}, {"start": 8.68, "end": 13.280000000000001, "text": " In this video, you learn how to construct a layer of neurons, and once you have that"}, {"start": 13.280000000000001, "end": 17.92, "text": " down, you'll be able to take those building blocks and put them together to form a large"}, {"start": 17.92, "end": 18.92, "text": " neural network."}, {"start": 18.92, "end": 22.44, "text": " Let's take a look at how a layer of neurons works."}, {"start": 22.44, "end": 28.36, "text": " Here's the example we had from the demand prediction example where we had four input"}, {"start": 28.36, "end": 35.92, "text": " features that were fed to this layer of three neurons in this hidden layer that then sends"}, {"start": 35.92, "end": 40.519999999999996, "text": " this output to this output layer with just one neuron."}, {"start": 40.519999999999996, "end": 47.08, "text": " Let's zoom in to the hidden layer to look at its computations."}, {"start": 47.08, "end": 52.82, "text": " This hidden layer inputs four numbers, and these four numbers are inputs to each of three"}, {"start": 52.82, "end": 61.08, "text": " neurons, and each of these three neurons is just implementing a little logistic regression"}, {"start": 61.08, "end": 65.2, "text": " unit or a little logistic regression function."}, {"start": 65.2, "end": 67.0, "text": " Take this first neuron."}, {"start": 67.0, "end": 75.44, "text": " It has two parameters, W and B, and in fact, to denote that this is the first hidden unit,"}, {"start": 75.44, "end": 83.52, "text": " I'm going to subscript this as W1B1, and what it does is it'll output some activation value"}, {"start": 83.52, "end": 96.12, "text": " A, which is G of W1 in a product of X plus B1, where this is the familiar Z value that"}, {"start": 96.12, "end": 104.16, "text": " you had learned about in logistic regression in the previous course, and G of Z is the"}, {"start": 104.16, "end": 111.75999999999999, "text": " familiar logistic function, one over one plus E to the negative Z, and so maybe this ends"}, {"start": 111.75999999999999, "end": 119.24, "text": " up being a number 0.3, and that's the activation value A of the first neuron."}, {"start": 119.24, "end": 124.19999999999999, "text": " To denote that this is the first neuron, I'm also going to add a subscript A1 over here,"}, {"start": 124.19999999999999, "end": 126.64, "text": " and so A1 may be a number like 0.3."}, {"start": 126.64, "end": 134.24, "text": " That's a 0.3 chance of this being highly affordable based on the input features."}, {"start": 134.24, "end": 136.28, "text": " Now let's look at the second neuron."}, {"start": 136.28, "end": 147.08, "text": " The second neuron has parameters W2 and B2, and this WB or W2B2 are the parameters of"}, {"start": 147.08, "end": 149.48, "text": " the second logistic unit."}, {"start": 149.48, "end": 159.95999999999998, "text": " So it computes A2 equals the logistic function G applied to W2 dot product X plus B2, and"}, {"start": 159.95999999999998, "end": 167.39999999999998, "text": " this may be some other number, say 0.7, because in this example there's a 0.7 chance that"}, {"start": 167.39999999999998, "end": 172.72, "text": " we think the potential buyers will be aware of this t-shirt."}, {"start": 172.72, "end": 178.76, "text": " And similarly, the third neuron has a third set of parameters, W3B3, and similarly computes"}, {"start": 178.76, "end": 186.67999999999998, "text": " an activation value A3 equals G of W3 dot product X plus B3, and that may be, say, 0.2."}, {"start": 186.67999999999998, "end": 194.94, "text": " So in this example, these three neurons output 0.3, 0.7, and 0.2, and this vector of three"}, {"start": 194.94, "end": 205.84, "text": " numbers becomes the vector of activation values A that is then passed to the final output"}, {"start": 205.84, "end": 208.39999999999998, "text": " layer of this neural network."}, {"start": 208.4, "end": 214.12, "text": " Now when you build neural networks with multiple layers, it will be useful to give the layers"}, {"start": 214.12, "end": 215.84, "text": " different numbers."}, {"start": 215.84, "end": 223.6, "text": " So by convention, this layer is called layer one of the neural network, and this layer"}, {"start": 223.6, "end": 230.24, "text": " is called layer two of the neural network, and the input layer is also sometimes called"}, {"start": 230.24, "end": 237.72, "text": " layer zero, and today there are neural networks that can have dozens or even hundreds of layers."}, {"start": 237.72, "end": 243.76, "text": " But in order to introduce notation to help us distinguish between the different layers,"}, {"start": 243.76, "end": 252.2, "text": " I'm going to use superscript square bracket one to index into different layers."}, {"start": 252.2, "end": 258.66, "text": " So in particular, A superscript in square brackets one, I'm going to use as a notation"}, {"start": 258.66, "end": 266.28, "text": " to denote the output of layer one of this hidden layer of this neural network, and similarly,"}, {"start": 266.28, "end": 273.35999999999996, "text": " W1B1 here are the parameters of the first unit in layer one of the neural network, so"}, {"start": 273.35999999999996, "end": 281.55999999999995, "text": " I'm also going to add a superscript in square brackets one here, and W2B2 are the parameters"}, {"start": 281.55999999999995, "end": 288.76, "text": " of the second hidden unit or the second hidden neuron in layer one, and so those parameters"}, {"start": 288.76, "end": 294.96, "text": " are also denoted here, W superscript square bracket one, like so."}, {"start": 294.96, "end": 300.68, "text": " And similarly, I can add superscript square brackets like so to denote that these are"}, {"start": 300.68, "end": 308.79999999999995, "text": " the activation values of the hidden units of layer one of this neural network."}, {"start": 308.79999999999995, "end": 315.24, "text": " I know maybe this notation is getting a little bit cluttered, but the thing to remember is"}, {"start": 315.24, "end": 322.59999999999997, "text": " whenever you see this superscript square bracket one, that just refers to a quantity that is"}, {"start": 322.6, "end": 329.16, "text": " associated with layer one of the neural network, and if you see superscript square bracket"}, {"start": 329.16, "end": 335.68, "text": " two, that refers to a quantity associated with layer two of the neural network, and"}, {"start": 335.68, "end": 340.76000000000005, "text": " similarly for other layers as well, including layer three, layer four, and so on for neural"}, {"start": 340.76000000000005, "end": 342.84000000000003, "text": " networks with more layers."}, {"start": 342.84000000000003, "end": 347.88, "text": " So that's the computation of layer one of this neural network."}, {"start": 347.88, "end": 355.6, "text": " This output is this activation vector, A superscript square bracket one, and I'm going to copy"}, {"start": 355.6, "end": 364.04, "text": " this over here because this output, A1, becomes the input to layer two."}, {"start": 364.04, "end": 369.92, "text": " So now let's zoom into the computation of layer two of this neural network, which is"}, {"start": 369.92, "end": 372.6, "text": " also the output layer."}, {"start": 372.6, "end": 378.48, "text": " So the input to layer two is the output of layer one."}, {"start": 378.48, "end": 389.36, "text": " So A1 is this vector 0.3, 0.7, 0.2 that we just computed on the previous part of the"}, {"start": 389.36, "end": 397.92, "text": " slide, and so because the output layer has just a single neuron, all it does is it computes"}, {"start": 397.92, "end": 405.6, "text": " a subscript one that is the output of its first and only neuron as G, the sigmoid function,"}, {"start": 405.6, "end": 412.40000000000003, "text": " applied to W subscript one in a product with A superscript square bracket one, so this"}, {"start": 412.40000000000003, "end": 418.36, "text": " is the input into this layer, and then plus B1."}, {"start": 418.36, "end": 425.40000000000003, "text": " Here this is the quantity Z that you're familiar with, and G as before is the sigmoid function"}, {"start": 425.4, "end": 433.44, "text": " that you apply to this, and if this results in a number, say 0.84, then that becomes the"}, {"start": 433.44, "end": 440.28, "text": " output of this output layer of the neural network, and in this example, because the"}, {"start": 440.28, "end": 445.84, "text": " output layer has just a single neuron, this output is just a scalar, it's a single number"}, {"start": 445.84, "end": 448.64, "text": " rather than a vector of numbers."}, {"start": 448.64, "end": 454.12, "text": " Sticking with our notational convention from before, we're going to use A superscript in"}, {"start": 454.12, "end": 460.72, "text": " square brackets two to denote the quantities associated with layer two of this neural network,"}, {"start": 460.72, "end": 467.56, "text": " so A superscript square bracket two is the output of this layer, and so I'm going to"}, {"start": 467.56, "end": 475.68, "text": " also copy this here as the final output of the neural network, and to make the notation"}, {"start": 475.68, "end": 481.84000000000003, "text": " consistent, you can also add these superscript square bracket twos to denote that these are"}, {"start": 481.84, "end": 489.4, "text": " the parameters and activation values associated with layer two of the neural network."}, {"start": 489.4, "end": 495.56, "text": " Once the neural network has computed A2, there's one final optional step that you can choose"}, {"start": 495.56, "end": 502.71999999999997, "text": " to implement or not, which is if you want a binary prediction, one or zero, is this"}, {"start": 502.71999999999997, "end": 508.88, "text": " a top seller, yes or no, is you can take the number A superscript square brackets two,"}, {"start": 508.88, "end": 518.04, "text": " subscript one, and this is the number 0.84 that we computed, and threshold is at 0.5,"}, {"start": 518.04, "end": 523.28, "text": " so if it's greater than 0.5, you can predict y hat equals one, and if it's less than 0.5,"}, {"start": 523.28, "end": 528.84, "text": " then predict y hat equals zero, and we saw this thresholding as well when we learned"}, {"start": 528.84, "end": 533.24, "text": " about logistic regression in the first course of the specialization."}, {"start": 533.24, "end": 538.6, "text": " So if you wish, this then gives you the final prediction y hat as either one or zero if"}, {"start": 538.6, "end": 542.4, "text": " you don't want just a probability of it being a top seller."}, {"start": 542.4, "end": 544.8000000000001, "text": " So that's how a neural network works."}, {"start": 544.8000000000001, "end": 549.6800000000001, "text": " Every layer inputs a vector of numbers and applies a bunch of logistic regression units"}, {"start": 549.6800000000001, "end": 555.1, "text": " to it and then computes another vector of numbers that then gets passed from layer to"}, {"start": 555.1, "end": 560.5600000000001, "text": " layer until you get the final output layers computation, which is a prediction of the"}, {"start": 560.5600000000001, "end": 567.88, "text": " neural network that you can either threshold at 0.5 or not to come up with the final prediction,"}, {"start": 567.88, "end": 573.4, "text": " and with that, let's go on to use this foundation we've built now to look at some even more"}, {"start": 573.4, "end": 580.08, "text": " complex, even larger neural network models, and I hope that by seeing more examples, this"}, {"start": 580.08, "end": 585.68, "text": " concept of layers and how to put them together to build a neural network will become even"}, {"start": 585.68, "end": 587.0, "text": " clearer."}, {"start": 587.0, "end": 598.64, "text": " So let's go on to the next video."}]