Neural network breaks if hidden layers are increased
I found my error. My early calculations only involved one hidden layer, however, I did not include the derivatives of previous weights when calculating the hidden_error. This means that in the scenario of a neural network of:

Input -&gt; H1 -&gt; H2 -&gt; Output


to calculate the backpropagation of the last step (Input &lt;- H1), I did not calculate derivatives of weights_hidden_output (H2 -> Output) and weights_hidden_to_hidden (H1 -> H2).