4.11 Neural network implementation in Python | Forward prop in a single layer-[ML| Andrew Ng]
 If you had to implement forward propagation yourself from scratch in Python, how would you go about doing so? In addition to gaining intuition about what's really going on in libraries like TensorFlow and PyTorch, if ever someday you decide you want to build something even better than TensorFlow and PyTorch, maybe now you'd have a better idea how. I don't really recommend doing this for most people, but maybe someday someone will come up with an even better framework than TensorFlow and PyTorch, and whoever does that may end up having to implement these things from scratch themselves. So let's take a look. On this slide, I'm going to go through quite a bit of code, and you'll see all this code again later in the optional lab as well as in the practice lab. So don't worry about having to take notes on every line of code or memorize every line of code. You see this code written down in the Jupyter Notebook in the lab, and the goal of this video is to just show you the code to make sure you can understand what it's doing so that when you go to the optional lab in the practice lab and see the code there, you know what to do. So don't worry about taking detailed notes on every line. If you can read through the code on this slide and understand what it's doing, that's all you need. So let's take a look at how you implement forward prop in a single layer. We're going to continue using the coffee roasting model shown here. And let's look at how you would take an input feature vector x and implement forward prop to get this output a2. In this Python implementation, I'm going to use one D arrays to represent all of these vectors and parameters, which is why there's only a single square bracket here. This is a one D array in Python rather than a 2D matrix, which is what we had when we had double square brackets. So the first value you need to compute is a superscript square bracket one, subscript one, which is the first activation value of a one. And that's G of this expression over here. So I'm going to use the convention on this slide that a term like w two one, I'm going to represent as a variable w two and then subscript one, this underscore one denotes subscript one. So w two means w superscript two in square brackets and then subscript one. So to compute a one one, we have parameters w one one and b one one, which are say one two and negative one. You would then compute z one one as the dot product between that parameter w one one and the input X and add it to be one one and then finally a one one is equal to G the safe point function applied to z one one. Next let's go on to compute a one two, which again by the convention I described here is going to be a one two written like that. So similar as what we did on the left, w one two is your two parameters minus three four, b one two is the term b one two over there. So you compute z as this term in the middle and then apply the safe point function and then you end up with a one two and finally you do the same thing to compute a one three. Now you've computed these three values a one one, a one two and a one three and we like to take these three numbers and group them together into an array to give you a one up here, which is the output of the first layer. And so you do that by grouping them together using a NumPy array as follows. So now you've computed a one. Let's implement the second layer as well to compute the output a two. So a two is computed using this expression and so we would have parameters w two one and b two one corresponding to these parameters and then you would compute z as the dot product between w two one and a one and add b two one and then apply the safe point function to get a two one and that's it. That's how you implement Ford prop using just Python and NumPy. Now there are a lot of expressions in this page of code that you just saw. Let's in the next video look at how you can simplify this to implement Ford prop for a more general neural network rather than hard coding it for every single neuron like we just did. So let's go see that in the next video.
[{"start": 0.0, "end": 7.6000000000000005, "text": " If you had to implement forward propagation yourself from scratch in Python, how would"}, {"start": 7.6000000000000005, "end": 9.72, "text": " you go about doing so?"}, {"start": 9.72, "end": 14.8, "text": " In addition to gaining intuition about what's really going on in libraries like TensorFlow"}, {"start": 14.8, "end": 20.16, "text": " and PyTorch, if ever someday you decide you want to build something even better than TensorFlow"}, {"start": 20.16, "end": 24.400000000000002, "text": " and PyTorch, maybe now you'd have a better idea how."}, {"start": 24.400000000000002, "end": 28.98, "text": " I don't really recommend doing this for most people, but maybe someday someone will come"}, {"start": 28.98, "end": 33.94, "text": " up with an even better framework than TensorFlow and PyTorch, and whoever does that may end"}, {"start": 33.94, "end": 37.38, "text": " up having to implement these things from scratch themselves."}, {"start": 37.38, "end": 39.28, "text": " So let's take a look."}, {"start": 39.28, "end": 44.28, "text": " On this slide, I'm going to go through quite a bit of code, and you'll see all this code"}, {"start": 44.28, "end": 48.66, "text": " again later in the optional lab as well as in the practice lab."}, {"start": 48.66, "end": 52.58, "text": " So don't worry about having to take notes on every line of code or memorize every line"}, {"start": 52.58, "end": 53.620000000000005, "text": " of code."}, {"start": 53.62, "end": 59.0, "text": " You see this code written down in the Jupyter Notebook in the lab, and the goal of this"}, {"start": 59.0, "end": 64.84, "text": " video is to just show you the code to make sure you can understand what it's doing so"}, {"start": 64.84, "end": 68.92, "text": " that when you go to the optional lab in the practice lab and see the code there, you know"}, {"start": 68.92, "end": 69.92, "text": " what to do."}, {"start": 69.92, "end": 72.56, "text": " So don't worry about taking detailed notes on every line."}, {"start": 72.56, "end": 77.08, "text": " If you can read through the code on this slide and understand what it's doing, that's all"}, {"start": 77.08, "end": 78.44, "text": " you need."}, {"start": 78.44, "end": 83.64, "text": " So let's take a look at how you implement forward prop in a single layer."}, {"start": 83.64, "end": 89.56, "text": " We're going to continue using the coffee roasting model shown here."}, {"start": 89.56, "end": 97.2, "text": " And let's look at how you would take an input feature vector x and implement forward prop"}, {"start": 97.2, "end": 100.08, "text": " to get this output a2."}, {"start": 100.08, "end": 106.6, "text": " In this Python implementation, I'm going to use one D arrays to represent all of these"}, {"start": 106.6, "end": 111.11999999999999, "text": " vectors and parameters, which is why there's only a single square bracket here."}, {"start": 111.11999999999999, "end": 116.91999999999999, "text": " This is a one D array in Python rather than a 2D matrix, which is what we had when we"}, {"start": 116.91999999999999, "end": 119.61999999999999, "text": " had double square brackets."}, {"start": 119.61999999999999, "end": 125.47999999999999, "text": " So the first value you need to compute is a superscript square bracket one, subscript"}, {"start": 125.47999999999999, "end": 130.28, "text": " one, which is the first activation value of a one."}, {"start": 130.28, "end": 133.51999999999998, "text": " And that's G of this expression over here."}, {"start": 133.52, "end": 141.20000000000002, "text": " So I'm going to use the convention on this slide that a term like w two one, I'm going"}, {"start": 141.20000000000002, "end": 148.88, "text": " to represent as a variable w two and then subscript one, this underscore one denotes"}, {"start": 148.88, "end": 149.88, "text": " subscript one."}, {"start": 149.88, "end": 156.56, "text": " So w two means w superscript two in square brackets and then subscript one."}, {"start": 156.56, "end": 167.48, "text": " So to compute a one one, we have parameters w one one and b one one, which are say one"}, {"start": 167.48, "end": 170.04, "text": " two and negative one."}, {"start": 170.04, "end": 178.56, "text": " You would then compute z one one as the dot product between that parameter w one one and"}, {"start": 178.56, "end": 188.2, "text": " the input X and add it to be one one and then finally a one one is equal to G the safe point"}, {"start": 188.2, "end": 192.16, "text": " function applied to z one one."}, {"start": 192.16, "end": 198.6, "text": " Next let's go on to compute a one two, which again by the convention I described here is"}, {"start": 198.6, "end": 203.1, "text": " going to be a one two written like that."}, {"start": 203.1, "end": 210.0, "text": " So similar as what we did on the left, w one two is your two parameters minus three four,"}, {"start": 210.0, "end": 213.72, "text": " b one two is the term b one two over there."}, {"start": 213.72, "end": 219.6, "text": " So you compute z as this term in the middle and then apply the safe point function and"}, {"start": 219.6, "end": 228.56, "text": " then you end up with a one two and finally you do the same thing to compute a one three."}, {"start": 228.56, "end": 236.68, "text": " Now you've computed these three values a one one, a one two and a one three and we like"}, {"start": 236.68, "end": 243.32, "text": " to take these three numbers and group them together into an array to give you a one up"}, {"start": 243.32, "end": 245.88, "text": " here, which is the output of the first layer."}, {"start": 245.88, "end": 251.84, "text": " And so you do that by grouping them together using a NumPy array as follows."}, {"start": 251.84, "end": 255.0, "text": " So now you've computed a one."}, {"start": 255.0, "end": 260.88, "text": " Let's implement the second layer as well to compute the output a two."}, {"start": 260.88, "end": 268.32, "text": " So a two is computed using this expression and so we would have parameters w two one"}, {"start": 268.32, "end": 275.4, "text": " and b two one corresponding to these parameters and then you would compute z as the dot product"}, {"start": 275.4, "end": 281.04, "text": " between w two one and a one and add b two one and then apply the safe point function"}, {"start": 281.04, "end": 284.96, "text": " to get a two one and that's it."}, {"start": 284.96, "end": 289.59999999999997, "text": " That's how you implement Ford prop using just Python and NumPy."}, {"start": 289.59999999999997, "end": 294.08, "text": " Now there are a lot of expressions in this page of code that you just saw."}, {"start": 294.08, "end": 298.79999999999995, "text": " Let's in the next video look at how you can simplify this to implement Ford prop for a"}, {"start": 298.79999999999995, "end": 303.4, "text": " more general neural network rather than hard coding it for every single neuron like we"}, {"start": 303.4, "end": 304.58, "text": " just did."}, {"start": 304.58, "end": 315.68, "text": " So let's go see that in the next video."}]