CosineAnnealingLR
class torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max, eta_min=0, last_epoch=- 1, verbose=False)
Set the learning rate of each parameter group using a cosine
   annealing schedule, where \eta_{max} is set to the initial lr and
   T_{cur} is the number of epochs since the last restart in SGDR:
  \begin{aligned}     \eta_t & = \eta_{min} +
  \frac{1}{2}(\eta_{max} - \eta_{min})\left(1     +
  \cos\left(\frac{T_{cur}}{T_{max}}\pi\right)\right),     &
  T_{cur} \neq (2k+1)T_{max}; \\     \eta_{t+1} & = \eta_{t} +
  \frac{1}{2}(\eta_{max} - \eta_{min})     \left(1 -
  \cos\left(\frac{1}{T_{max}}\pi\right)\right),     & T_{cur} =
  (2k+1)T_{max}. \end{aligned}

When last_epoch=-1, sets initial lr as lr. Notice that because the
   schedule is defined recursively, the learning rate can be
   simultaneously modified outside this scheduler by other operators.
   If the learning rate is set solely by this scheduler, the learning