4.8 TensorFlow implementation | Inference in Code --[Machine Learning | Andrew Ng]
 TensorFlow is one of the leading frameworks for implementing deep learning algorithms. When I'm building projects, TensorFlow is actually the tool that I use the most often and the other popular tool is PyTorch. But we're going to focus in this specialization on TensorFlow. In this video, let's take a look at how you can implement inference in code using TensorFlow. Let's dive in. One of the remarkable things about neural networks is the same algorithm can be applied to so many different applications. So in order both for this video and in some of the labs for you to see what a neural network is doing, I'm going to use another example to illustrate inference. So sometimes I do like to roast coffee beans myself at home. My favorite is actually Colombian coffee beans. So kind of learning algorithm help optimize the quality of the beans you get from a roasting process like this. When you're roasting coffee, two parameters you get to control are the temperature at which are heating up the raw coffee beans to turn them into nicely roasted coffee beans, as well as the duration or how long you're going to roast the beans. And in this slightly simplified example, we've created the data sets of different temperatures and different durations, as well as labels showing whether the coffee you roasted is good tasting coffee, where cross here, the positive class, y equals one, corresponds to good coffee, and oh, the negative class corresponds to bad coffee. So it looks like a reasonable way to think of this data set is if you cook it at too low a temperature, it doesn't get roasted and ends up undercooked. If you cook it not for long enough, the duration is too short. There's also not a nicely roasted set of beans. And finally, if you were to cook it either for too long or for too high a temperature, then you end up with overcooked beans, they're a little bit burnt beans. And so that's not good coffee either. There's only points within this little triangle here that corresponds to good coffee. This example is simplified a bit from actual coffee roasting. Even though this example is a simplified one for the purpose of illustration, there have actually been serious projects using machine learning to optimize coffee roasting as well. So the task is given a feature vector x with both temperature and duration, say 200 degrees Celsius for 17 minutes, how can we do inference in a neural network to get it to tell us whether or not this temperature and duration setting will result in good coffee or not? It looks like this. We're going to set x to be an array of two numbers, the input features 200 degrees Celsius and 17 minutes. And this here, layer one equals dense units, three activation equals sigmoid, creates a hidden layer of neurons with three hidden units and using as the activation function, the sigmoid function. And dense here is just the name of this layer. And then finally, to compute the activation values a one, you would write a one equals layer one applied to the input features x. Then you create layer one as this first hidden layer of the neural network as dense, open prints units three, that means three units or three hidden units in this layer using as the activation function, the sigmoid function. And dense is another name for the layers of a neural network that we've learned about so far. And as you learn more about neural networks, you learn about other types of layers as well. But for now, we just use the dense layer, which is the layer type you've learned about in the last few videos for all of our examples. So next, you compute a one by taking layer one, which is actually a function and applying this function layer one to the values of x. So that's how you get a one, which is going to be a list of three numbers because they one had three units. And so a one here may just for the sake of illustration be point 2.7.3. Next, for the second hidden layer, layer two would be dense of. Now this time as one unit and again to signal and activation function. And you can then compute a two by applying this layer two function to the activation values from layer one to a one. And that will give you the value of a two, which for the sake of illustration is maybe zero point eight. Finally, if you wish to threshold is at 0.5, then you can just test if a two is greater than or equal to 0.5 and set y hat equals to one or zero positive or negative class accordingly. So that's how you do inference in the neural network using TensorFlow. There are some additional details that I didn't go over here, such as how to load the TensorFlow library and how to also load the parameters W and B of the neural network. But we'll go over that in the lab. So please be sure to take a look at the lab. But these are the key steps for for propagation and how you compute a one and a two and optionally threshold a two. Let's look at one more example. And we're going to go back to the handwritten digit classification problem. So in this example, X is a list of the pixel intensity values. So X is equal to a non-pi array of this list of pixel intensity values. And then to initialize and carry out one step before propagation, layer one is a dense layer with 25 units and a sigmoid activation function. And you then compute a one equals the layer one function applied to X to build and carry out inference through the second layer. Similarly, you set up layer two as follows and then compute a two as layer two applied to a one. And then finally, layer three is the third and final dense layer. And then finally, you can optionally threshold a three to come up with a binary prediction for Y hat. So that's the syntax for carrying out inference in TensorFlow. One thing I briefly alluded to is the structure of the non-pi arrays. TensorFlow treats data in a certain way that is important to get right. So in the next video, let's take a look at how TensorFlow handles data.
[{"start": 0.0, "end": 6.22, "text": " TensorFlow is one of the leading frameworks for implementing deep learning algorithms."}, {"start": 6.22, "end": 10.86, "text": " When I'm building projects, TensorFlow is actually the tool that I use the most often"}, {"start": 10.86, "end": 13.94, "text": " and the other popular tool is PyTorch."}, {"start": 13.94, "end": 18.3, "text": " But we're going to focus in this specialization on TensorFlow."}, {"start": 18.3, "end": 24.1, "text": " In this video, let's take a look at how you can implement inference in code using TensorFlow."}, {"start": 24.1, "end": 25.1, "text": " Let's dive in."}, {"start": 25.1, "end": 30.060000000000002, "text": " One of the remarkable things about neural networks is the same algorithm can be applied"}, {"start": 30.060000000000002, "end": 33.36, "text": " to so many different applications."}, {"start": 33.36, "end": 39.6, "text": " So in order both for this video and in some of the labs for you to see what a neural network"}, {"start": 39.6, "end": 46.16, "text": " is doing, I'm going to use another example to illustrate inference."}, {"start": 46.16, "end": 50.540000000000006, "text": " So sometimes I do like to roast coffee beans myself at home."}, {"start": 50.540000000000006, "end": 53.96, "text": " My favorite is actually Colombian coffee beans."}, {"start": 53.96, "end": 58.980000000000004, "text": " So kind of learning algorithm help optimize the quality of the beans you get from a roasting"}, {"start": 58.980000000000004, "end": 61.06, "text": " process like this."}, {"start": 61.06, "end": 65.94, "text": " When you're roasting coffee, two parameters you get to control are the temperature at"}, {"start": 65.94, "end": 72.64, "text": " which are heating up the raw coffee beans to turn them into nicely roasted coffee beans,"}, {"start": 72.64, "end": 76.7, "text": " as well as the duration or how long you're going to roast the beans."}, {"start": 76.7, "end": 83.22, "text": " And in this slightly simplified example, we've created the data sets of different temperatures"}, {"start": 83.22, "end": 90.38, "text": " and different durations, as well as labels showing whether the coffee you roasted is"}, {"start": 90.38, "end": 96.14, "text": " good tasting coffee, where cross here, the positive class, y equals one, corresponds"}, {"start": 96.14, "end": 102.22, "text": " to good coffee, and oh, the negative class corresponds to bad coffee."}, {"start": 102.22, "end": 109.02, "text": " So it looks like a reasonable way to think of this data set is if you cook it at too"}, {"start": 109.02, "end": 114.17999999999999, "text": " low a temperature, it doesn't get roasted and ends up undercooked."}, {"start": 114.17999999999999, "end": 118.5, "text": " If you cook it not for long enough, the duration is too short."}, {"start": 118.5, "end": 121.67999999999999, "text": " There's also not a nicely roasted set of beans."}, {"start": 121.67999999999999, "end": 126.89999999999999, "text": " And finally, if you were to cook it either for too long or for too high a temperature,"}, {"start": 126.89999999999999, "end": 130.34, "text": " then you end up with overcooked beans, they're a little bit burnt beans."}, {"start": 130.34, "end": 132.84, "text": " And so that's not good coffee either."}, {"start": 132.84, "end": 139.3, "text": " There's only points within this little triangle here that corresponds to good coffee."}, {"start": 139.3, "end": 143.74, "text": " This example is simplified a bit from actual coffee roasting."}, {"start": 143.74, "end": 150.1, "text": " Even though this example is a simplified one for the purpose of illustration, there have"}, {"start": 150.1, "end": 155.46, "text": " actually been serious projects using machine learning to optimize coffee roasting as well."}, {"start": 155.46, "end": 163.06, "text": " So the task is given a feature vector x with both temperature and duration, say 200 degrees"}, {"start": 163.06, "end": 171.06, "text": " Celsius for 17 minutes, how can we do inference in a neural network to get it to tell us whether"}, {"start": 171.06, "end": 176.9, "text": " or not this temperature and duration setting will result in good coffee or not?"}, {"start": 176.9, "end": 179.58, "text": " It looks like this."}, {"start": 179.58, "end": 189.82000000000002, "text": " We're going to set x to be an array of two numbers, the input features 200 degrees Celsius"}, {"start": 189.82000000000002, "end": 192.34, "text": " and 17 minutes."}, {"start": 192.34, "end": 199.82000000000002, "text": " And this here, layer one equals dense units, three activation equals sigmoid, creates a"}, {"start": 199.82000000000002, "end": 206.02, "text": " hidden layer of neurons with three hidden units and using as the activation function,"}, {"start": 206.02, "end": 207.98000000000002, "text": " the sigmoid function."}, {"start": 207.98, "end": 211.06, "text": " And dense here is just the name of this layer."}, {"start": 211.06, "end": 219.33999999999997, "text": " And then finally, to compute the activation values a one, you would write a one equals"}, {"start": 219.33999999999997, "end": 223.66, "text": " layer one applied to the input features x."}, {"start": 223.66, "end": 231.62, "text": " Then you create layer one as this first hidden layer of the neural network as dense, open"}, {"start": 231.62, "end": 237.9, "text": " prints units three, that means three units or three hidden units in this layer using"}, {"start": 237.9, "end": 241.46, "text": " as the activation function, the sigmoid function."}, {"start": 241.46, "end": 246.58, "text": " And dense is another name for the layers of a neural network that we've learned about"}, {"start": 246.58, "end": 247.82, "text": " so far."}, {"start": 247.82, "end": 253.34, "text": " And as you learn more about neural networks, you learn about other types of layers as well."}, {"start": 253.34, "end": 257.5, "text": " But for now, we just use the dense layer, which is the layer type you've learned about"}, {"start": 257.5, "end": 260.96, "text": " in the last few videos for all of our examples."}, {"start": 260.96, "end": 268.62, "text": " So next, you compute a one by taking layer one, which is actually a function and applying"}, {"start": 268.62, "end": 271.41999999999996, "text": " this function layer one to the values of x."}, {"start": 271.41999999999996, "end": 277.26, "text": " So that's how you get a one, which is going to be a list of three numbers because they"}, {"start": 277.26, "end": 279.09999999999997, "text": " one had three units."}, {"start": 279.09999999999997, "end": 284.85999999999996, "text": " And so a one here may just for the sake of illustration be point 2.7.3."}, {"start": 284.85999999999996, "end": 290.46, "text": " Next, for the second hidden layer, layer two would be dense of."}, {"start": 290.46, "end": 295.21999999999997, "text": " Now this time as one unit and again to signal and activation function."}, {"start": 295.21999999999997, "end": 300.14, "text": " And you can then compute a two by applying this layer two function to the activation"}, {"start": 300.14, "end": 303.5, "text": " values from layer one to a one."}, {"start": 303.5, "end": 307.21999999999997, "text": " And that will give you the value of a two, which for the sake of illustration is maybe"}, {"start": 307.21999999999997, "end": 308.9, "text": " zero point eight."}, {"start": 308.9, "end": 316.08, "text": " Finally, if you wish to threshold is at 0.5, then you can just test if a two is greater"}, {"start": 316.08, "end": 321.5, "text": " than or equal to 0.5 and set y hat equals to one or zero positive or negative class"}, {"start": 321.5, "end": 323.46, "text": " accordingly."}, {"start": 323.46, "end": 327.78, "text": " So that's how you do inference in the neural network using TensorFlow."}, {"start": 327.78, "end": 332.62, "text": " There are some additional details that I didn't go over here, such as how to load the TensorFlow"}, {"start": 332.62, "end": 339.82, "text": " library and how to also load the parameters W and B of the neural network."}, {"start": 339.82, "end": 341.47999999999996, "text": " But we'll go over that in the lab."}, {"start": 341.47999999999996, "end": 344.06, "text": " So please be sure to take a look at the lab."}, {"start": 344.06, "end": 350.66, "text": " But these are the key steps for for propagation and how you compute a one and a two and optionally"}, {"start": 350.66, "end": 353.06, "text": " threshold a two."}, {"start": 353.06, "end": 354.74, "text": " Let's look at one more example."}, {"start": 354.74, "end": 360.26, "text": " And we're going to go back to the handwritten digit classification problem."}, {"start": 360.26, "end": 365.04, "text": " So in this example, X is a list of the pixel intensity values."}, {"start": 365.04, "end": 370.06, "text": " So X is equal to a non-pi array of this list of pixel intensity values."}, {"start": 370.06, "end": 377.98, "text": " And then to initialize and carry out one step before propagation, layer one is a dense layer"}, {"start": 377.98, "end": 382.1, "text": " with 25 units and a sigmoid activation function."}, {"start": 382.1, "end": 388.3, "text": " And you then compute a one equals the layer one function applied to X to build and carry"}, {"start": 388.3, "end": 391.78, "text": " out inference through the second layer."}, {"start": 391.78, "end": 400.73999999999995, "text": " Similarly, you set up layer two as follows and then compute a two as layer two applied to a one."}, {"start": 400.73999999999995, "end": 405.94, "text": " And then finally, layer three is the third and final dense layer."}, {"start": 405.94, "end": 410.5, "text": " And then finally, you can optionally threshold a three to come up with a binary prediction"}, {"start": 410.5, "end": 412.65999999999997, "text": " for Y hat."}, {"start": 412.65999999999997, "end": 417.46, "text": " So that's the syntax for carrying out inference in TensorFlow."}, {"start": 417.46, "end": 422.58, "text": " One thing I briefly alluded to is the structure of the non-pi arrays."}, {"start": 422.58, "end": 427.35999999999996, "text": " TensorFlow treats data in a certain way that is important to get right."}, {"start": 427.36, "end": 448.26, "text": " So in the next video, let's take a look at how TensorFlow handles data."}]