Why does decreasing the SGD learning rate cause a massive increase in accuracy?
With a higher learning rate, you take bigger steps towards the solution. However, when you are close, you might jump over the solution and then the next step, you jump over it again causing an oscillation around the solution.
Now, if you lower the learning rate correctly, you will stop the oscillation and continue towards the solution once again. That is, until you start oscillating again.
To keep in mind is that a larger learning rate can jump over smaller local minima and help you find a better minima, which it can't jump over.
Also, it is generally the training error that becomes better, and the validation error becomes worse as you start to overfit on the training data.