The activation function is used as a scaling method to keep values within a range where they remain useful. It is important to note that activation functions must generally be non-linear and must always have a known derivative. In this work, a sigmoid activation function (equation 3.3) is used in hidden layers due to its easily calculated derivative, and the softmax activation function is used in the output layer (equation 3.4).