5.2 Neural Network Training | Training Details --[Machine Learning | Andrew Ng]
 Let's take a look at the details of what the TensorFlow code for training a neural network is actually doing. Let's dive in. Before looking at the details of training a neural network, let's recall how you had trained a logistic regression model in the previous course. Step 1 of building a logistic regression model was you would specify how to compute the output given the input v to the x and the parameters w and b. So in the first course we said the logistic regression function predicts f of x is equal to g, the sigmoid function applied to w dot product x plus b, which was the sigmoid function applied to w dot x plus b. So if z is the dot product of w of x plus b, then f of x is 1 over 1 plus e to the negative z. So that was the first step, where to specify what is the input to output function of logistic regression, and that depends on both the input x and the parameters of the model. The second step we had to do to train the logistic regression model was to specify the loss function and also the cost function. So you may recall that the loss function said if logistic regression outputs f of x and the ground truth label, the actual label in the training set was y, then the loss on that single training example was negative y log f of x minus 1 minus y times log of 1 minus f of x. So this was a measure of how well is logistic regression doing on a single training example x, y. Given this definition of a loss function, we then define the cost function, and the cost function was a function of the parameters w and b, and that was just the average that is taking an average over all m training examples of the loss function computed on the m training examples x1, y1, through xm, ym. And remember that in the condition we're using, the loss function is a function of the output of the learning algorithm and the ground truth label as computed over a single training example, whereas the cost function j is an average of the loss function computed over your entire training set. So that was step two of what we did when building up logistic regression. And then the third and final step to train the logistic regression model was to use an algorithm, specifically gradient descent, to minimize that cost function j of w, b to minimize it as a function of the parameters w and b. And we minimize the cost j as a function of the parameters using gradient descent, where w is updated as w minus the learning rate alpha times the derivative of j with respect to w, and b similarly is updated as b minus the learning rate alpha times the derivative of j with respect to b. So with these three steps, step one, specifying how to compute the outputs given the input x in parameters, step two, specifying loss in cost, and step three, minimize the cost function, we trained logistic regression. The same three steps is how we can train a neural network in TensorFlow. Now let's look at how these three steps map to training a neural network. We'll go over this in greater detail on the next three slides, but really briefly, step one of specifying how to compute the output given the input x in parameters w and b, that's done with this code snippet, which should be familiar from last week of specifying the neural network, and this was actually enough to specify the computations needed in forward propagation or for the inference algorithm, for example. The second step is to compile the model and to tell it what loss you want to use. And here's a code that you use to specify this loss function, which is the binary cross NGP loss function. And once you specify this loss, taking an average over the entire training set also gives you the cost function for the neural network. And then step three is to call function to try to minimize the cost as a function of the parameters of the neural network. Let's look in greater detail in these three steps in the context of training a neural network. The first step, specify how to compute the output given the input x in parameters w and b. This code snippet specifies the entire architecture of the neural network. It tells you that there are 25 hidden units in the first hidden layer, then 15 in the next one, and then one output unit, and that we're using the sig one activation value. And so based on this code snippet, we know also what are the parameters, w1, b1 of the first layer, parameters of the second layer, and parameters of the third layer. So this code snippet specifies the entire architecture of the neural network and therefore tells TensorFlow everything it needs in order to compute the output x as a function. In order to compute the output a3 or f of x as a function of the input x and the parameters here we have written wl and bl. Let's go on to step two. In the second step, you have to specify what is the loss function, and that will also define the cost function we use to train the neural network. So for the MNIST 01 digit classification problem is a binary classification problem, and the most common by far loss function to use is this one. It's actually the same loss function as what we had for logistic regression is negative y log f of x minus one minus y times log one minus f of x, where y is the ground truth label, sometimes also called the target label y, and f of x is now the output of the neural network. So in the terminology of TensorFlow, this loss function is called binary cross entropy, and the syntax is to ask TensorFlow to compile the neural network using this loss function. And another historical note, Keras was originally a library that had developed independently of TensorFlow, it was actually a totally separate project from TensorFlow, but eventually it got merged into TensorFlow, which is why we have tf.keraslibrary.losses.the name of this loss function. And by the way, I don't always remember the names of all the loss functions in TensorFlow, but I just do a quick web search myself to find the right name, and then I plug that into my code. Having specified the loss with respect to a single training example, TensorFlow knows that the costs you want to minimize is then the average, taking the average over all m training examples of the loss on all of the training examples. And optimizing this cost function will result in fitting the neural network to your binary classification data. In case you want to solve a regression problem rather than a classification problem, you can also tell TensorFlow to compile your model using a different loss function. For example, if you have a regression problem, and if you want to minimize the squared error loss, so here is the squared error loss, the loss with respect to if you're learning algorithm outputs f of x with a target or ground truth label of y, that's one half of the squared error, then you can use this loss function in TensorFlow, which is to use the maybe more intuitively named mean squared error loss function, and then TensorFlow will try to minimize the mean squared error. In this expression, I'm using J of capital W comma capital B to denote the cost function. The cost function is a function of all of the parameters in the neural network. So you can think of capital W as including W1, W2, W3, so all the W parameters in the entire neural network and B as including B1, B2, and B3. So if you are optimizing the cost function with respect to W and B, you'd be trying to optimize it with respect to all of the parameters in the neural network. And up on top as well, I had written f of x as the output of the neural network, but you want, you can also write f of WB if we want to emphasize that the output of the neural network as a function of x depends on all the parameters in all the layers of the neural network. So that's the loss function and the cost function. And in TensorFlow, this is called the binary cross entropy loss function. Where does that name come from? Well, it turns out in statistics, this function on top is called the cross entropy loss function. So that's what cross entropy means. And the word binary just re-emphasizes or points out that this is a binary classification problem because each image is either a zero or a one. Finally, you will ask TensorFlow to minimize the cost function. You might remember the gradient descent algorithm from the first course. If you are using gradient descent to train the parameters of a neural network, then you will repeatedly for every layer L and for every unit J, update WLJ according to WLJ minus the learning rate alpha times the partial derivative with respect to that parameter of the cost function J of WB. And similarly for the parameters B as well. And after doing, say, 100 iterations of gradient descent, hopefully you get to a good value of the parameters. So in order to use gradient descent, the key thing you need to compute is these partial derivative terms. And what TensorFlow does, and in fact what is standard in neural network training, is to use an algorithm called backpropagation in order to compute these partial derivative terms. TensorFlow can do all of these things for you. It implements backpropagation all within this function called fit. So all you have to do is call model.fit x, y as your training set and tell it to do so for 100 iterations or 100 epochs. In fact, what you see later is that TensorFlow can use an algorithm that is even a little bit faster than gradient descent. And you see more about that later this week as well. Now I know that we're relying heavily on the TensorFlow library in order to implement a neural network. One pattern I've seen across multiple ideas is as the technology evolves, libraries become more mature and most engineers will use libraries rather than implement code from scratch. And there have been many other examples of this in the history of computing. Once many, many decades ago programmers had to implement their own sorting function from scratch. But now sorting libraries are quite mature that you probably call someone else's sorting function rather than implement it yourself unless you're taking a computing classes, ask you to do it as an exercise. And today if you want to compute the square root of a number, like what is the square root of 7? Well once programmers had to write their own code to compute this, but now pretty much everyone just calls a library to take square roots or matrix operations such as multiplying two matrices together. So when deep learning was younger and less mature, many developers, including me, were implementing things from scratch using Python or C++ or some other library. But today deep learning libraries have matured enough that most developers will use these libraries and in fact most commercial implementations of neural networks today use a library like TensorFlow or PyTorch. But as I've mentioned, it's still useful to understand how they work under the hood so that if something unexpected happens, which still does with today's libraries, you have a better chance of knowing how to fix it. Now that you know how to train a basic neural network, also called a multilayer perceptron, there are some things you can change about the neural network that will make it even more powerful. In the next video, let's take a look at how you can swap in different activation functions as an alternative to the sigmoid activation function we've been using. This will make your neural networks work even much better. So let's go take a look at that in the next video.
[{"start": 0.0, "end": 6.96, "text": " Let's take a look at the details of what the TensorFlow code for training a neural network"}, {"start": 6.96, "end": 7.96, "text": " is actually doing."}, {"start": 7.96, "end": 9.64, "text": " Let's dive in."}, {"start": 9.64, "end": 14.58, "text": " Before looking at the details of training a neural network, let's recall how you had"}, {"start": 14.58, "end": 19.76, "text": " trained a logistic regression model in the previous course."}, {"start": 19.76, "end": 26.6, "text": " Step 1 of building a logistic regression model was you would specify how to compute the output"}, {"start": 26.6, "end": 31.0, "text": " given the input v to the x and the parameters w and b."}, {"start": 31.0, "end": 37.760000000000005, "text": " So in the first course we said the logistic regression function predicts f of x is equal"}, {"start": 37.760000000000005, "end": 43.92, "text": " to g, the sigmoid function applied to w dot product x plus b, which was the sigmoid function"}, {"start": 43.92, "end": 46.84, "text": " applied to w dot x plus b."}, {"start": 46.84, "end": 58.92, "text": " So if z is the dot product of w of x plus b, then f of x is 1 over 1 plus e to the negative"}, {"start": 58.92, "end": 60.760000000000005, "text": " z."}, {"start": 60.760000000000005, "end": 66.48, "text": " So that was the first step, where to specify what is the input to output function of logistic"}, {"start": 66.48, "end": 72.48, "text": " regression, and that depends on both the input x and the parameters of the model."}, {"start": 72.48, "end": 78.08, "text": " The second step we had to do to train the logistic regression model was to specify the"}, {"start": 78.08, "end": 81.26, "text": " loss function and also the cost function."}, {"start": 81.26, "end": 89.52000000000001, "text": " So you may recall that the loss function said if logistic regression outputs f of x and"}, {"start": 89.52000000000001, "end": 95.80000000000001, "text": " the ground truth label, the actual label in the training set was y, then the loss on that"}, {"start": 95.8, "end": 103.28, "text": " single training example was negative y log f of x minus 1 minus y times log of 1 minus"}, {"start": 103.28, "end": 104.52, "text": " f of x."}, {"start": 104.52, "end": 110.67999999999999, "text": " So this was a measure of how well is logistic regression doing on a single training example"}, {"start": 110.67999999999999, "end": 115.0, "text": " x, y."}, {"start": 115.0, "end": 121.44, "text": " Given this definition of a loss function, we then define the cost function, and the"}, {"start": 121.44, "end": 127.92, "text": " cost function was a function of the parameters w and b, and that was just the average that"}, {"start": 127.92, "end": 134.2, "text": " is taking an average over all m training examples of the loss function computed on the m training"}, {"start": 134.2, "end": 139.88, "text": " examples x1, y1, through xm, ym."}, {"start": 139.88, "end": 147.76, "text": " And remember that in the condition we're using, the loss function is a function of the output"}, {"start": 147.76, "end": 154.07999999999998, "text": " of the learning algorithm and the ground truth label as computed over a single training example,"}, {"start": 154.07999999999998, "end": 159.64, "text": " whereas the cost function j is an average of the loss function computed over your entire"}, {"start": 159.64, "end": 161.35999999999999, "text": " training set."}, {"start": 161.35999999999999, "end": 166.56, "text": " So that was step two of what we did when building up logistic regression."}, {"start": 166.56, "end": 172.32, "text": " And then the third and final step to train the logistic regression model was to use an"}, {"start": 172.32, "end": 179.64, "text": " algorithm, specifically gradient descent, to minimize that cost function j of w, b to"}, {"start": 179.64, "end": 183.88, "text": " minimize it as a function of the parameters w and b."}, {"start": 183.88, "end": 188.95999999999998, "text": " And we minimize the cost j as a function of the parameters using gradient descent, where"}, {"start": 188.95999999999998, "end": 196.24, "text": " w is updated as w minus the learning rate alpha times the derivative of j with respect"}, {"start": 196.24, "end": 206.04000000000002, "text": " to w, and b similarly is updated as b minus the learning rate alpha times the derivative"}, {"start": 206.04000000000002, "end": 209.12, "text": " of j with respect to b."}, {"start": 209.12, "end": 213.36, "text": " So with these three steps, step one, specifying how to compute the outputs given the input"}, {"start": 213.36, "end": 218.0, "text": " x in parameters, step two, specifying loss in cost, and step three, minimize the cost"}, {"start": 218.0, "end": 221.4, "text": " function, we trained logistic regression."}, {"start": 221.4, "end": 227.96, "text": " The same three steps is how we can train a neural network in TensorFlow."}, {"start": 227.96, "end": 233.12, "text": " Now let's look at how these three steps map to training a neural network."}, {"start": 233.12, "end": 238.72, "text": " We'll go over this in greater detail on the next three slides, but really briefly, step"}, {"start": 238.72, "end": 244.0, "text": " one of specifying how to compute the output given the input x in parameters w and b, that's"}, {"start": 244.0, "end": 249.32, "text": " done with this code snippet, which should be familiar from last week of specifying the"}, {"start": 249.32, "end": 254.6, "text": " neural network, and this was actually enough to specify the computations needed in forward"}, {"start": 254.6, "end": 257.71999999999997, "text": " propagation or for the inference algorithm, for example."}, {"start": 257.71999999999997, "end": 263.76, "text": " The second step is to compile the model and to tell it what loss you want to use."}, {"start": 263.76, "end": 269.32, "text": " And here's a code that you use to specify this loss function, which is the binary cross"}, {"start": 269.32, "end": 273.2, "text": " NGP loss function."}, {"start": 273.2, "end": 278.15999999999997, "text": " And once you specify this loss, taking an average over the entire training set also"}, {"start": 278.16, "end": 281.0, "text": " gives you the cost function for the neural network."}, {"start": 281.0, "end": 286.56, "text": " And then step three is to call function to try to minimize the cost as a function of"}, {"start": 286.56, "end": 288.96000000000004, "text": " the parameters of the neural network."}, {"start": 288.96000000000004, "end": 293.52000000000004, "text": " Let's look in greater detail in these three steps in the context of training a neural"}, {"start": 293.52000000000004, "end": 296.04, "text": " network."}, {"start": 296.04, "end": 300.08000000000004, "text": " The first step, specify how to compute the output given the input x in parameters w and"}, {"start": 300.08000000000004, "end": 301.08000000000004, "text": " b."}, {"start": 301.08000000000004, "end": 305.02000000000004, "text": " This code snippet specifies the entire architecture of the neural network."}, {"start": 305.02, "end": 309.96, "text": " It tells you that there are 25 hidden units in the first hidden layer, then 15 in the"}, {"start": 309.96, "end": 315.32, "text": " next one, and then one output unit, and that we're using the sig one activation value."}, {"start": 315.32, "end": 320.03999999999996, "text": " And so based on this code snippet, we know also what are the parameters, w1, b1 of the"}, {"start": 320.03999999999996, "end": 324.78, "text": " first layer, parameters of the second layer, and parameters of the third layer."}, {"start": 324.78, "end": 330.28, "text": " So this code snippet specifies the entire architecture of the neural network and therefore"}, {"start": 330.28, "end": 336.65999999999997, "text": " tells TensorFlow everything it needs in order to compute the output x as a function."}, {"start": 336.65999999999997, "end": 344.52, "text": " In order to compute the output a3 or f of x as a function of the input x and the parameters"}, {"start": 344.52, "end": 349.47999999999996, "text": " here we have written wl and bl."}, {"start": 349.47999999999996, "end": 351.32, "text": " Let's go on to step two."}, {"start": 351.32, "end": 356.03999999999996, "text": " In the second step, you have to specify what is the loss function, and that will also define"}, {"start": 356.03999999999996, "end": 359.88, "text": " the cost function we use to train the neural network."}, {"start": 359.88, "end": 369.68, "text": " So for the MNIST 01 digit classification problem is a binary classification problem, and the"}, {"start": 369.68, "end": 373.64, "text": " most common by far loss function to use is this one."}, {"start": 373.64, "end": 379.0, "text": " It's actually the same loss function as what we had for logistic regression is negative"}, {"start": 379.0, "end": 387.0, "text": " y log f of x minus one minus y times log one minus f of x, where y is the ground truth"}, {"start": 387.0, "end": 392.72, "text": " label, sometimes also called the target label y, and f of x is now the output of the neural"}, {"start": 392.72, "end": 393.96, "text": " network."}, {"start": 393.96, "end": 401.52, "text": " So in the terminology of TensorFlow, this loss function is called binary cross entropy,"}, {"start": 401.52, "end": 408.16, "text": " and the syntax is to ask TensorFlow to compile the neural network using this loss function."}, {"start": 408.16, "end": 414.16, "text": " And another historical note, Keras was originally a library that had developed independently"}, {"start": 414.16, "end": 418.72, "text": " of TensorFlow, it was actually a totally separate project from TensorFlow, but eventually it"}, {"start": 418.72, "end": 425.52000000000004, "text": " got merged into TensorFlow, which is why we have tf.keraslibrary.losses.the name of this"}, {"start": 425.52000000000004, "end": 427.44000000000005, "text": " loss function."}, {"start": 427.44000000000005, "end": 432.52000000000004, "text": " And by the way, I don't always remember the names of all the loss functions in TensorFlow,"}, {"start": 432.52000000000004, "end": 436.36, "text": " but I just do a quick web search myself to find the right name, and then I plug that"}, {"start": 436.36, "end": 438.36, "text": " into my code."}, {"start": 438.36, "end": 443.8, "text": " Having specified the loss with respect to a single training example, TensorFlow knows"}, {"start": 443.8, "end": 448.92, "text": " that the costs you want to minimize is then the average, taking the average over all m"}, {"start": 448.92, "end": 453.44, "text": " training examples of the loss on all of the training examples."}, {"start": 453.44, "end": 459.92, "text": " And optimizing this cost function will result in fitting the neural network to your binary"}, {"start": 459.92, "end": 461.44, "text": " classification data."}, {"start": 461.44, "end": 468.04, "text": " In case you want to solve a regression problem rather than a classification problem, you"}, {"start": 468.04, "end": 474.8, "text": " can also tell TensorFlow to compile your model using a different loss function."}, {"start": 474.8, "end": 481.64000000000004, "text": " For example, if you have a regression problem, and if you want to minimize the squared error"}, {"start": 481.64000000000004, "end": 487.24, "text": " loss, so here is the squared error loss, the loss with respect to if you're learning algorithm"}, {"start": 487.24, "end": 492.84000000000003, "text": " outputs f of x with a target or ground truth label of y, that's one half of the squared"}, {"start": 492.84, "end": 500.56, "text": " error, then you can use this loss function in TensorFlow, which is to use the maybe more"}, {"start": 500.56, "end": 505.34, "text": " intuitively named mean squared error loss function, and then TensorFlow will try to"}, {"start": 505.34, "end": 507.52, "text": " minimize the mean squared error."}, {"start": 507.52, "end": 514.72, "text": " In this expression, I'm using J of capital W comma capital B to denote the cost function."}, {"start": 514.72, "end": 520.04, "text": " The cost function is a function of all of the parameters in the neural network."}, {"start": 520.04, "end": 529.16, "text": " So you can think of capital W as including W1, W2, W3, so all the W parameters in the"}, {"start": 529.16, "end": 536.88, "text": " entire neural network and B as including B1, B2, and B3."}, {"start": 536.88, "end": 543.5999999999999, "text": " So if you are optimizing the cost function with respect to W and B, you'd be trying to"}, {"start": 543.5999999999999, "end": 548.36, "text": " optimize it with respect to all of the parameters in the neural network."}, {"start": 548.36, "end": 554.2, "text": " And up on top as well, I had written f of x as the output of the neural network, but"}, {"start": 554.2, "end": 559.84, "text": " you want, you can also write f of WB if we want to emphasize that the output of the neural"}, {"start": 559.84, "end": 564.44, "text": " network as a function of x depends on all the parameters in all the layers of the neural"}, {"start": 564.44, "end": 565.64, "text": " network."}, {"start": 565.64, "end": 568.48, "text": " So that's the loss function and the cost function."}, {"start": 568.48, "end": 575.0, "text": " And in TensorFlow, this is called the binary cross entropy loss function."}, {"start": 575.0, "end": 576.2, "text": " Where does that name come from?"}, {"start": 576.2, "end": 582.12, "text": " Well, it turns out in statistics, this function on top is called the cross entropy loss function."}, {"start": 582.12, "end": 584.0400000000001, "text": " So that's what cross entropy means."}, {"start": 584.0400000000001, "end": 589.58, "text": " And the word binary just re-emphasizes or points out that this is a binary classification"}, {"start": 589.58, "end": 594.84, "text": " problem because each image is either a zero or a one."}, {"start": 594.84, "end": 599.86, "text": " Finally, you will ask TensorFlow to minimize the cost function."}, {"start": 599.86, "end": 605.1800000000001, "text": " You might remember the gradient descent algorithm from the first course."}, {"start": 605.18, "end": 609.56, "text": " If you are using gradient descent to train the parameters of a neural network, then you"}, {"start": 609.56, "end": 619.12, "text": " will repeatedly for every layer L and for every unit J, update WLJ according to WLJ"}, {"start": 619.12, "end": 626.4399999999999, "text": " minus the learning rate alpha times the partial derivative with respect to that parameter"}, {"start": 626.4399999999999, "end": 629.88, "text": " of the cost function J of WB."}, {"start": 629.88, "end": 634.38, "text": " And similarly for the parameters B as well."}, {"start": 634.38, "end": 641.16, "text": " And after doing, say, 100 iterations of gradient descent, hopefully you get to a good value"}, {"start": 641.16, "end": 643.24, "text": " of the parameters."}, {"start": 643.24, "end": 649.12, "text": " So in order to use gradient descent, the key thing you need to compute is these partial"}, {"start": 649.12, "end": 651.4, "text": " derivative terms."}, {"start": 651.4, "end": 656.32, "text": " And what TensorFlow does, and in fact what is standard in neural network training, is"}, {"start": 656.32, "end": 662.44, "text": " to use an algorithm called backpropagation in order to compute these partial derivative"}, {"start": 662.44, "end": 663.44, "text": " terms."}, {"start": 663.44, "end": 666.7600000000001, "text": " TensorFlow can do all of these things for you."}, {"start": 666.7600000000001, "end": 671.1600000000001, "text": " It implements backpropagation all within this function called fit."}, {"start": 671.1600000000001, "end": 677.7600000000001, "text": " So all you have to do is call model.fit x, y as your training set and tell it to do so"}, {"start": 677.7600000000001, "end": 681.8800000000001, "text": " for 100 iterations or 100 epochs."}, {"start": 681.8800000000001, "end": 686.6400000000001, "text": " In fact, what you see later is that TensorFlow can use an algorithm that is even a little"}, {"start": 686.6400000000001, "end": 688.6, "text": " bit faster than gradient descent."}, {"start": 688.6, "end": 691.5200000000001, "text": " And you see more about that later this week as well."}, {"start": 691.52, "end": 697.28, "text": " Now I know that we're relying heavily on the TensorFlow library in order to implement a"}, {"start": 697.28, "end": 699.04, "text": " neural network."}, {"start": 699.04, "end": 705.84, "text": " One pattern I've seen across multiple ideas is as the technology evolves, libraries become"}, {"start": 705.84, "end": 712.6, "text": " more mature and most engineers will use libraries rather than implement code from scratch."}, {"start": 712.6, "end": 717.48, "text": " And there have been many other examples of this in the history of computing."}, {"start": 717.48, "end": 722.4, "text": " Once many, many decades ago programmers had to implement their own sorting function from"}, {"start": 722.4, "end": 723.9200000000001, "text": " scratch."}, {"start": 723.9200000000001, "end": 728.76, "text": " But now sorting libraries are quite mature that you probably call someone else's sorting"}, {"start": 728.76, "end": 733.12, "text": " function rather than implement it yourself unless you're taking a computing classes,"}, {"start": 733.12, "end": 736.04, "text": " ask you to do it as an exercise."}, {"start": 736.04, "end": 740.76, "text": " And today if you want to compute the square root of a number, like what is the square"}, {"start": 740.76, "end": 742.32, "text": " root of 7?"}, {"start": 742.32, "end": 747.28, "text": " Well once programmers had to write their own code to compute this, but now pretty much"}, {"start": 747.28, "end": 754.4, "text": " everyone just calls a library to take square roots or matrix operations such as multiplying"}, {"start": 754.4, "end": 757.16, "text": " two matrices together."}, {"start": 757.16, "end": 762.6, "text": " So when deep learning was younger and less mature, many developers, including me, were"}, {"start": 762.6, "end": 767.9599999999999, "text": " implementing things from scratch using Python or C++ or some other library."}, {"start": 767.9599999999999, "end": 773.36, "text": " But today deep learning libraries have matured enough that most developers will use these"}, {"start": 773.36, "end": 779.0, "text": " libraries and in fact most commercial implementations of neural networks today use a library like"}, {"start": 779.0, "end": 780.88, "text": " TensorFlow or PyTorch."}, {"start": 780.88, "end": 785.0, "text": " But as I've mentioned, it's still useful to understand how they work under the hood so"}, {"start": 785.0, "end": 789.42, "text": " that if something unexpected happens, which still does with today's libraries, you have"}, {"start": 789.42, "end": 791.84, "text": " a better chance of knowing how to fix it."}, {"start": 791.84, "end": 798.44, "text": " Now that you know how to train a basic neural network, also called a multilayer perceptron,"}, {"start": 798.44, "end": 802.58, "text": " there are some things you can change about the neural network that will make it even"}, {"start": 802.58, "end": 804.48, "text": " more powerful."}, {"start": 804.48, "end": 809.5, "text": " In the next video, let's take a look at how you can swap in different activation functions"}, {"start": 809.5, "end": 814.5600000000001, "text": " as an alternative to the sigmoid activation function we've been using."}, {"start": 814.5600000000001, "end": 818.12, "text": " This will make your neural networks work even much better."}, {"start": 818.12, "end": 833.76, "text": " So let's go take a look at that in the next video."}]