How does one use torch.optim.lr_scheduler.OneCycleLR()?
I wanted to use torch.optim.lr_scheduler.OneCycleLR() while training. Can some kindly explain to me how to use it?
What i got from the documentation was that it should be called after each train_batch.
My confusions are as follows:

Does the max_lr parameter has to be same with the optimizer lr parameter?

Can this scheduler be used with Adam optimizer. How is the momentum calculated then?

Let’s say i trained my model for some number of epochs at a stretch now, i wanted to train for some more epochs. Would i have to reset the the scheduler?


Can anybody provide me a sort of a toy example/training loop that implements this scheduler?
I am kind of new to deep learning &amp; PyTorch so my question might be somewhat silly.

You might get some use out of this thread: How to use Pytorch OneCycleLR in a training loop (and optimizer/scheduler interactions)?
But to address your points:

Does the max_lr parameter has to be same with the optimizer lr parameter? No, this is the max or highest value -- a hyperparameter that you will experiment with. Notice in the paper the use of max_lr: https://arxiv.org/pdf/1708.07120.pdf

Can this scheduler be used with Adam optimizer. How is the momentum calculated then? Yes.

Let’s say i trained my model for some number of epochs at a stretch now, i wanted to train for some more epochs. Would i have to reset the the scheduler?  Depends, are you loading the model from a saved checkpoint or not? Check PyTorch's tutorials: https://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#sphx-glr-beginner-blitz-neural-networks-tutorial-py


