class_docstr: Linearly increases learning rate from 0 to 1 over `warmup` fraction of training steps.
  If `cycles` (default=1.) is different from default, learning rate follows `cycles` times a cosine decaying
  learning rate (with hard restarts).
code: def __init__(self, warmup=0.002, t_total=-1, cycles=1.0, **kw):
      super(WarmupCosineWithHardRestartsSchedule, self).__init__(
          warmup=warmup, t_total=t_total, cycles=cycles, **kw
      )
      assert cycles >= 1.0
  
imports: import math
  import torch
  from torch.optim import Optimizer
  from torch.optim.optimizer import required
  from torch.nn.utils import clip_grad_norm_
  import logging
  import abc
  import sys
label_desc: Use image node_id 1 to create a new WarmupCosineWithHardRestartsSchedule object from inherited base classes: WarmupCosineSchedule with example: obj = WarmupCosineWithHardRestartsSchedule(warmup, t_total, cycles)