4.10 TensorFlow implementation | Building a neural network --[Machine Learning | Andrew Ng]
 So you've seen a bunch of TensorFlow code by now, learned about how to build a layer in TensorFlow, how to do forward prop through a single layer in TensorFlow, and also learned about data in TensorFlow. Let's put it all together and talk about how to build a neural network in TensorFlow. This is also the last video on TensorFlow for this week, and in this video, you also learned about a different way of building a neural network that will be even a little bit simpler than what you've seen so far. So let's dive in. What you saw previously was if you want to do forward prop, you initialize the data x, create layer one like so, then compute A1, then create layer two, and compute A2. So this was an explicit way of carrying out forward prop one layer of computation at the time. It turns out that TensorFlow has a different way of implementing forward prop as well as learning. Let me show you a different way of building a neural network in TensorFlow, which is that same as before, you're going to create layer one and create layer two. But now, instead of you manually taking the data and passing it to layer one, and then taking the activations from layer one and passing it to layer two, we can instead tell TensorFlow that we would like it to take layer one and layer two and string them together to form a neural network. That's what the sequential function in TensorFlow does, which is it says, dear TensorFlow, please create a neural network for me by sequentially stringing together these two layers that I just created. It turns out that with the sequential framework, TensorFlow can do a lot of work for you. Let's say you have a training set like this on the left. This is for the coffee example. You can then take the training data's inputs X and put them into a NumPy array. This here is a four by two matrix, and the target labels Y can then be written as follows. This is just a four dimensional array. Y, the set of targets, can then be stored as a 1D array like this, 1 0 0 1, corresponding to the four training examples. It turns out that given the data X and Y stored in this matrix X and this array Y, if you want to train this neural network, all you need to do is call two functions. You need to call model.compile with some parameters. We'll talk more about this next week, so don't worry about it for now. And then you need to call model.fit X Y, which tells TensorFlow to take this neural network that it created by sequentially stringing together layers one and two, and to train it on the data X and Y. But we'll learn the details of how to do this next week. And then finally, how do you do inference on this neural network? How do you do forward prop? If you have a new example, say X new, which is NP array with these two features, then to carry out forward prop, instead of having to do it one layer at a time yourself, you just have to call model predict on X new, and this will output the corresponding value of A2 for you, given this input value of X. So model predict carries out forward propagation or carries out inference for you using this neural network that you compiled using the sequential function. Now, I want to take these three lines of code on top and just simplify it a little bit further, which is when coding in TensorFlow, by convention, we don't explicitly assign the two layers to two variables, layer one and layer two as follows, but by convention, I would usually just write the code like this. What we say the model is a sequential model of a few layers strung together sequentially, where the first layer, layer one, is a dense layer with three units and activation of sigmoid, and the second layer is a dense layer with one unit and again a sigmoid activation function. So if you look at others TensorFlow code, you often see it look more like this, rather than having an explicit assignment to these layer one and layer two variables. And so that's it. This is pretty much the code you need in order to train as well as to inference on a neural network in TensorFlow. Where again, we'll talk more about the training bits of this, the compound, the compile and the fit function next week. Let's redo this for the digit classification example as well. So previously we had X is this input, layer one is a layer, A1 equals layer one applied to X and so on through layer two and layer three in order to try to classify a digit. With this new coding convention, with using TensorFlow's sequential function, you can instead specify what are layer one, layer two, layer three, and tell TensorFlow to string the layers together for you into a neural network. And same as before, you can then store the data in a matrix and run the compile function and fit the model as follows. Again, more on this next week. And finally, to do inference or to make predictions, you can use model predict on X new. And similar to what you saw before with the coffee classification network, by convention, instead of assigning layer one, layer two, layer three explicitly like this, we would more calmly just take these layers and put them directly into the sequential function to end up with this more compact code where you just tell TensorFlow, create a model for me that sequentially strings together these three layers and then the rest of the code works same as before. So that's how you would build a neural network in TensorFlow. Now I know that when you're learning about these techniques, sometimes someone may ask you to, hey, implement these five lines of code, and then you type five lines of code and then someone says, congratulations with just five lines of code, you've built this crazy complicated state of the art neural network. And sometimes that makes you wonder what exactly did I do with just these five lines of code. One thing I want you to take away from the machine learning specialization is the ability to use cutting edge libraries, like TensorFlow to do your work efficiently. But I don't really want you to just call five lines of code and not really also know what the code is actually doing underneath the hood. So in the next video, I'd like to go back and share with you how you can implement from scratch by yourself for propagation in Python so that you can understand the whole thing for yourself. In practice, most machine learning engineers don't actually implement for a problem in Python that often. We just use libraries like TensorFlow and PyTorch. But because I want you to understand how these algorithms work yourself, so that if something goes wrong, you can think through for yourself what you might need to change, what's likely to work, what's less likely to work. Let's also go through what it would take for you to implement for propagation from scratch. Because that way, even when you're calling a library and having it run efficiently and do great things in your application, I want you in the back of your mind to also have that deeper understanding of what your code is actually doing. So with that, let's go on to the next video.
[{"start": 0.0, "end": 5.6000000000000005, "text": " So you've seen a bunch of TensorFlow code by now, learned about how to build a layer"}, {"start": 5.6000000000000005, "end": 11.28, "text": " in TensorFlow, how to do forward prop through a single layer in TensorFlow, and also learned"}, {"start": 11.28, "end": 13.88, "text": " about data in TensorFlow."}, {"start": 13.88, "end": 19.52, "text": " Let's put it all together and talk about how to build a neural network in TensorFlow."}, {"start": 19.52, "end": 24.12, "text": " This is also the last video on TensorFlow for this week, and in this video, you also"}, {"start": 24.12, "end": 28.8, "text": " learned about a different way of building a neural network that will be even a little"}, {"start": 28.8, "end": 31.48, "text": " bit simpler than what you've seen so far."}, {"start": 31.48, "end": 33.0, "text": " So let's dive in."}, {"start": 33.0, "end": 40.760000000000005, "text": " What you saw previously was if you want to do forward prop, you initialize the data x,"}, {"start": 40.760000000000005, "end": 46.6, "text": " create layer one like so, then compute A1, then create layer two, and compute A2."}, {"start": 46.6, "end": 53.0, "text": " So this was an explicit way of carrying out forward prop one layer of computation at the"}, {"start": 53.0, "end": 55.480000000000004, "text": " time."}, {"start": 55.48, "end": 63.279999999999994, "text": " It turns out that TensorFlow has a different way of implementing forward prop as well as"}, {"start": 63.279999999999994, "end": 64.28, "text": " learning."}, {"start": 64.28, "end": 70.32, "text": " Let me show you a different way of building a neural network in TensorFlow, which is that"}, {"start": 70.32, "end": 75.12, "text": " same as before, you're going to create layer one and create layer two."}, {"start": 75.12, "end": 81.67999999999999, "text": " But now, instead of you manually taking the data and passing it to layer one, and then"}, {"start": 81.68, "end": 86.80000000000001, "text": " taking the activations from layer one and passing it to layer two, we can instead tell"}, {"start": 86.80000000000001, "end": 93.32000000000001, "text": " TensorFlow that we would like it to take layer one and layer two and string them together"}, {"start": 93.32000000000001, "end": 95.64000000000001, "text": " to form a neural network."}, {"start": 95.64000000000001, "end": 102.4, "text": " That's what the sequential function in TensorFlow does, which is it says, dear TensorFlow, please"}, {"start": 102.4, "end": 108.2, "text": " create a neural network for me by sequentially stringing together these two layers that I"}, {"start": 108.2, "end": 110.64000000000001, "text": " just created."}, {"start": 110.64, "end": 118.04, "text": " It turns out that with the sequential framework, TensorFlow can do a lot of work for you."}, {"start": 118.04, "end": 120.68, "text": " Let's say you have a training set like this on the left."}, {"start": 120.68, "end": 123.16, "text": " This is for the coffee example."}, {"start": 123.16, "end": 131.4, "text": " You can then take the training data's inputs X and put them into a NumPy array."}, {"start": 131.4, "end": 142.0, "text": " This here is a four by two matrix, and the target labels Y can then be written as follows."}, {"start": 142.0, "end": 145.52, "text": " This is just a four dimensional array."}, {"start": 145.52, "end": 153.0, "text": " Y, the set of targets, can then be stored as a 1D array like this, 1 0 0 1, corresponding"}, {"start": 153.0, "end": 155.52, "text": " to the four training examples."}, {"start": 155.52, "end": 164.48000000000002, "text": " It turns out that given the data X and Y stored in this matrix X and this array Y, if you"}, {"start": 164.48000000000002, "end": 170.24, "text": " want to train this neural network, all you need to do is call two functions."}, {"start": 170.24, "end": 174.60000000000002, "text": " You need to call model.compile with some parameters."}, {"start": 174.60000000000002, "end": 178.68, "text": " We'll talk more about this next week, so don't worry about it for now."}, {"start": 178.68, "end": 186.16, "text": " And then you need to call model.fit X Y, which tells TensorFlow to take this neural network"}, {"start": 186.16, "end": 192.52, "text": " that it created by sequentially stringing together layers one and two, and to train"}, {"start": 192.52, "end": 196.52, "text": " it on the data X and Y."}, {"start": 196.52, "end": 201.4, "text": " But we'll learn the details of how to do this next week."}, {"start": 201.4, "end": 204.92000000000002, "text": " And then finally, how do you do inference on this neural network?"}, {"start": 204.92000000000002, "end": 206.60000000000002, "text": " How do you do forward prop?"}, {"start": 206.6, "end": 213.56, "text": " If you have a new example, say X new, which is NP array with these two features, then"}, {"start": 213.56, "end": 219.32, "text": " to carry out forward prop, instead of having to do it one layer at a time yourself, you"}, {"start": 219.32, "end": 227.56, "text": " just have to call model predict on X new, and this will output the corresponding value"}, {"start": 227.56, "end": 234.2, "text": " of A2 for you, given this input value of X."}, {"start": 234.2, "end": 239.82, "text": " So model predict carries out forward propagation or carries out inference for you using this"}, {"start": 239.82, "end": 244.2, "text": " neural network that you compiled using the sequential function."}, {"start": 244.2, "end": 252.12, "text": " Now, I want to take these three lines of code on top and just simplify it a little bit further,"}, {"start": 252.12, "end": 259.58, "text": " which is when coding in TensorFlow, by convention, we don't explicitly assign the two layers"}, {"start": 259.58, "end": 265.8, "text": " to two variables, layer one and layer two as follows, but by convention, I would usually"}, {"start": 265.8, "end": 267.8, "text": " just write the code like this."}, {"start": 267.8, "end": 273.2, "text": " What we say the model is a sequential model of a few layers strung together sequentially,"}, {"start": 273.2, "end": 279.91999999999996, "text": " where the first layer, layer one, is a dense layer with three units and activation of sigmoid,"}, {"start": 279.91999999999996, "end": 286.2, "text": " and the second layer is a dense layer with one unit and again a sigmoid activation function."}, {"start": 286.2, "end": 291.4, "text": " So if you look at others TensorFlow code, you often see it look more like this, rather"}, {"start": 291.4, "end": 298.76, "text": " than having an explicit assignment to these layer one and layer two variables."}, {"start": 298.76, "end": 300.84, "text": " And so that's it."}, {"start": 300.84, "end": 307.96, "text": " This is pretty much the code you need in order to train as well as to inference on a neural"}, {"start": 307.96, "end": 309.8, "text": " network in TensorFlow."}, {"start": 309.8, "end": 314.48, "text": " Where again, we'll talk more about the training bits of this, the compound, the compile and"}, {"start": 314.48, "end": 317.32, "text": " the fit function next week."}, {"start": 317.32, "end": 321.88, "text": " Let's redo this for the digit classification example as well."}, {"start": 321.88, "end": 329.0, "text": " So previously we had X is this input, layer one is a layer, A1 equals layer one applied"}, {"start": 329.0, "end": 335.40000000000003, "text": " to X and so on through layer two and layer three in order to try to classify a digit."}, {"start": 335.40000000000003, "end": 341.68, "text": " With this new coding convention, with using TensorFlow's sequential function, you can"}, {"start": 341.68, "end": 348.48, "text": " instead specify what are layer one, layer two, layer three, and tell TensorFlow to string"}, {"start": 348.48, "end": 352.24, "text": " the layers together for you into a neural network."}, {"start": 352.24, "end": 360.4, "text": " And same as before, you can then store the data in a matrix and run the compile function"}, {"start": 360.4, "end": 362.36, "text": " and fit the model as follows."}, {"start": 362.36, "end": 365.28000000000003, "text": " Again, more on this next week."}, {"start": 365.28, "end": 374.59999999999997, "text": " And finally, to do inference or to make predictions, you can use model predict on X new."}, {"start": 374.59999999999997, "end": 380.08, "text": " And similar to what you saw before with the coffee classification network, by convention,"}, {"start": 380.08, "end": 384.28, "text": " instead of assigning layer one, layer two, layer three explicitly like this, we would"}, {"start": 384.28, "end": 389.79999999999995, "text": " more calmly just take these layers and put them directly into the sequential function"}, {"start": 389.79999999999995, "end": 394.47999999999996, "text": " to end up with this more compact code where you just tell TensorFlow, create a model for"}, {"start": 394.48, "end": 399.04, "text": " me that sequentially strings together these three layers and then the rest of the code"}, {"start": 399.04, "end": 401.20000000000005, "text": " works same as before."}, {"start": 401.20000000000005, "end": 405.0, "text": " So that's how you would build a neural network in TensorFlow."}, {"start": 405.0, "end": 408.76, "text": " Now I know that when you're learning about these techniques, sometimes someone may ask"}, {"start": 408.76, "end": 413.64000000000004, "text": " you to, hey, implement these five lines of code, and then you type five lines of code"}, {"start": 413.64000000000004, "end": 417.40000000000003, "text": " and then someone says, congratulations with just five lines of code, you've built this"}, {"start": 417.40000000000003, "end": 420.74, "text": " crazy complicated state of the art neural network."}, {"start": 420.74, "end": 426.44, "text": " And sometimes that makes you wonder what exactly did I do with just these five lines of code."}, {"start": 426.44, "end": 430.46000000000004, "text": " One thing I want you to take away from the machine learning specialization is the ability"}, {"start": 430.46000000000004, "end": 435.86, "text": " to use cutting edge libraries, like TensorFlow to do your work efficiently."}, {"start": 435.86, "end": 441.56, "text": " But I don't really want you to just call five lines of code and not really also know what"}, {"start": 441.56, "end": 445.0, "text": " the code is actually doing underneath the hood."}, {"start": 445.0, "end": 449.76, "text": " So in the next video, I'd like to go back and share with you how you can implement from"}, {"start": 449.76, "end": 454.71999999999997, "text": " scratch by yourself for propagation in Python so that you can understand the whole thing"}, {"start": 454.71999999999997, "end": 456.44, "text": " for yourself."}, {"start": 456.44, "end": 461.68, "text": " In practice, most machine learning engineers don't actually implement for a problem in"}, {"start": 461.68, "end": 463.59999999999997, "text": " Python that often."}, {"start": 463.59999999999997, "end": 466.59999999999997, "text": " We just use libraries like TensorFlow and PyTorch."}, {"start": 466.59999999999997, "end": 471.36, "text": " But because I want you to understand how these algorithms work yourself, so that if something"}, {"start": 471.36, "end": 475.8, "text": " goes wrong, you can think through for yourself what you might need to change, what's likely"}, {"start": 475.8, "end": 478.32, "text": " to work, what's less likely to work."}, {"start": 478.32, "end": 483.92, "text": " Let's also go through what it would take for you to implement for propagation from scratch."}, {"start": 483.92, "end": 488.76, "text": " Because that way, even when you're calling a library and having it run efficiently and"}, {"start": 488.76, "end": 493.76, "text": " do great things in your application, I want you in the back of your mind to also have"}, {"start": 493.76, "end": 498.26, "text": " that deeper understanding of what your code is actually doing."}, {"start": 498.26, "end": 509.24, "text": " So with that, let's go on to the next video."}]