Is stable learning preferable to jumps in accuracy/loss
There is an approach to machine learning, called Simulated Annealing, which varies the rate: starting from a large rate, it is slowly reduced over time. The general idea is that the initial larger rate will cover a broader range, while the increasingly lower rate then produces a less 'erratic' climb towards a maximum.
If you only use a low rate, you risk getting stuck in a local maximum, while too large a rate will not find the best solution but might end up close to one. Adjusting the rate gives you the best of both.