OneCycleLR
class torch.optim.lr_scheduler.OneCycleLR(optimizer, max_lr, total_steps=None, epochs=None, steps_per_epoch=None, pct_start=0.3, anneal_strategy='cos', cycle_momentum=True, base_momentum=0.85, max_momentum=0.95, div_factor=25.0, final_div_factor=10000.0, three_phase=False, last_epoch=- 1, verbose=False)
Sets the learning rate of each parameter group according to the
   1cycle learning rate policy. The 1cycle policy anneals the learning
   rate from an initial learning rate to some maximum learning rate
   and then from that maximum learning rate to some minimum learning
   rate much lower than the initial learning rate. This policy was
   initially described in the paper Super-Convergence: Very Fast
   Training of Neural Networks Using Large Learning Rates.
The 1cycle learning rate policy changes the learning rate after
   every batch. step should be called after a batch has been used
   for training.
This scheduler is not chainable.