How do you know when your learning rate should be reduced?
The question is more of a general question on tips on how to know if it should be reduced. If you aren't getting good results, would a reduction in learning rate help?
Generally you have a start and and end and step down miles stones. How do you know the learning rate should not just go lower and lower? &quot;Just try it&quot; is probably the answer.
There was a paper where it equated learning rate with complexity. So as a model runs it's learning rate needs to be reduced to build more complex data patterns.
Any insights would be great.
Thanks.

Often it makes sense to reduce the learning rate when no more improvements can be achieved with the currently set learning rate. The whole process can also be automated. If you use Pytorch have a look here: https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
But it is important to find a good learning rate for the beginning. I often try it out for a few epochs. If you don't get good results, it may be due to the learning rate, but it doesn't have to be. It can also be too small as well as too large. If the loss values fluctuate strongly, it is probably too large and if they do not go down too small.
