class_docstr: Set the learning rate of each parameter group using a cosine annealing
  schedule, where :math:`\eta_{max}` is set to the initial lr and
  :math:`T_{cur}` is the number of epochs since the last restart in SGDR:
  
  .. math::
  
      \eta_t = \eta_{min} + \frac{1}{2}(\eta_{max} - \eta_{min})(1 +
      \cos(\frac{T_{cur}}{T_{max}}\pi))
  
  When last_epoch=-1, sets initial lr as lr.
  
  It has been proposed in
  `SGDR: Stochastic Gradient Descent with Warm Restarts`_. Note that this only
  implements the cosine annealing part of SGDR, and not the restarts.
  
  Args:
      optimizer (Optimizer): Wrapped optimizer.
      T_max (int): Maximum number of iterations.
      eta_min (float): Minimum learning rate. Default: 0.
      last_epoch (int): The index of last epoch. Default: -1.
  
  .. _SGDR\: Stochastic Gradient Descent with Warm Restarts:
      https://arxiv.org/abs/1608.03983
class_docstr_tok: ["Set","the","learning","rate","of","each","parameter","group","using","a","cosine","annealing","schedule",",","where",":","math",":","`","\\eta_","{","max","}","`","is","set","to","the","initial","lr","and",":","math",":","`","T_","{","cur","}","`","is","the","number","of","epochs","since","the","last","restart","in","SGDR",":","..","math",":",":","\\eta_t","=","\\eta_","{","min","}","+","\\frac","{","1","}","{","2","}","(","\\eta_","{","max","}","-","\\eta_","{","min","}",")","(","1","+","\\cos","(","\\frac","{","T_","{","cur","}","}","{","T_","{","max","}","}","\\pi",")",")","When","last_epoch=-1",",","sets","initial","lr","as","lr",".","It","has","been","proposed","in","`","SGDR",":","Stochastic","Gradient","Descent","with","Warm","Restarts","`","_",".","Note","that","this","only","implements","the","cosine","annealing","part","of","SGDR",",","and","not","the","restarts",".","Args",":","optimizer","(","Optimizer",")",":","Wrapped","optimizer",".","T_max","(","int",")",":","Maximum","number","of","iterations",".","eta_min","(","float",")",":","Minimum","learning","rate",".","Default",":","0.","last_epoch","(","int",")",":","The","index","of","last","epoch",".","Default",":","-1",".","..","_SGDR\\",":","Stochastic","Gradient","Descent","with","Warm","Restarts",":","https",":","\/\/arxiv.org\/abs\/1608.03983"]
code: def __init__(self, optimizer, T_max, gamma, eta_min=0, last_epoch=-1):
      self.gamma = gamma
      self.T_max = T_max
      self.eta_min = eta_min
      super(CosineAnnealingLRWithDecay, self).__init__(
          optimizer, last_epoch
      )
  
imports: import math
  import warnings
  import numpy
  from torch import nn
  from torch.optim.optimizer import Optimizer
  from torch.optim.lr_scheduler import _LRScheduler, LambdaLR, ExponentialLR, CyclicLR, MultiStepLR, ReduceLROnPlateau, CosineAnnealingLR, CosineAnnealingWarmRestarts
label_desc: Use image node_id 1 to create a new CosineAnnealingLRWithDecay object from inherited base classes: _LRScheduler with example: obj = CosineAnnealingLRWithDecay(optimizer, T_max, gamma, eta_min, last_epoch)