The training process of neural networks is similar to that of the supervised machine learning models as we have discussed in Section 2.1. For example, we can learn the neural network parameters 𝜃 to maximize the likelihood function 𝑝(𝑿, 𝒀|𝜃), which is equivalent to minimizing a squared error for regression or a cross-entropy error for classification. The key distinction between training neural networks and machine learning models is that the loss surface of neural networks is nonconvex due to the presence of nonlinearity, so convergence to the global minimum is not guaranteed. Gradient-based methods, including mini-batch gradient descent and stochastic gradient descent, are often used to optimize the neural network parameters with respect to the loss function by back-propagating the gradients (68).