Why isn't the loss of my neural network reduced after 2500 iterations?
You may try to adjust the learning rate first. As the learning rate has a great effect on changing the weights and the bias value.
See if the results has changed after adjusting the learning rate.