There are several common issues that remain to be accounted for; firstly, with a fixed learning rate, the model is adjusted in a fixed step size, however, when the model approaches a minimum that is surrounded by large gradients, it is possible that the model never reaches this minimum. To account for this, an annealing rate 𝐾 𝑎 may be applied that reduces the learning rate 𝐾 𝑙 as the number of training steps increases, shown in equation 3.11 below, where 𝑁 denotes the number of training steps performed.