class_docstr: A LearningRateSchedule that uses a cosine decay schedule with restarts.
  
  See [Loshchilov & Hutter, ICLR2016](https://arxiv.org/abs/1608.03983),
  SGDR: Stochastic Gradient Descent with Warm Restarts.
  
  When training a model, it is often useful to lower the learning rate as
  the training progresses. This schedule applies a cosine decay function with
  restarts to an optimizer step, given a provided initial learning rate.
  It requires a `step` value to compute the decayed learning rate. You can
  just pass a TensorFlow variable that you increment at each training step.
  
  The schedule a 1-arg callable that produces a decayed learning
  rate when passed the current optimizer step. This can be useful for changing
  the learning rate value across different invocations of optimizer functions.
  
  The learning rate multiplier first decays
  from 1 to `alpha` for `first_decay_steps` steps. Then, a warm
  restart is performed. Each new warm restart runs for `t_mul` times more
  steps and with `m_mul` times smaller initial learning rate.
  
  Example usage:
  ```python
  first_decay_steps = 1000
  lr_decayed_fn = (
    tf.keras.optimizers.schedules.CosineDecayRestarts(
        initial_learning_rate,
        first_decay_steps))
  ```
  
  You can pass this schedule directly into a `tf.keras.optimizers.Optimizer`
  as the learning rate. The learning rate schedule is also serializable and
  deserializable using `tf.keras.optimizers.schedules.serialize` and
  `tf.keras.optimizers.schedules.deserialize`.
  
  Returns:
    A 1-arg callable learning rate schedule that takes the current optimizer
    step and outputs the decayed learning rate, a scalar `Tensor` of the same
    type as `initial_learning_rate`.
docstr: Applies cosine decay with restarts to the learning rate.
  
  Args:
    initial_learning_rate: A scalar `float32` or `float64` Tensor or a Python
      number. The initial learning rate.
    first_decay_steps: A scalar `int32` or `int64` `Tensor` or a Python
      number. Number of steps to decay over.
    t_mul: A scalar `float32` or `float64` `Tensor` or a Python number.
      Used to derive the number of iterations in the i-th period
    m_mul: A scalar `float32` or `float64` `Tensor` or a Python number.
      Used to derive the initial learning rate of the i-th period:
    alpha: A scalar `float32` or `float64` Tensor or a Python number.
      Minimum learning rate value as a fraction of the initial_learning_rate.
    name: String. Optional name of the operation.  Defaults to 'SGDRDecay'.
docstr_tok: ["Applies","cosine","decay","with","restarts","to","the","learning","rate",".","Args",":","initial_learning_rate",":","A","scalar","`","float32","`","or","`","float64","`","Tensor","or","a","Python","number",".","The","initial","learning","rate",".","first_decay_steps",":","A","scalar","`","int32","`","or","`","int64","`","`","Tensor","`","or","a","Python","number",".","Number","of","steps","to","decay","over",".","t_mul",":","A","scalar","`","float32","`","or","`","float64","`","`","Tensor","`","or","a","Python","number",".","Used","to","derive","the","number","of","iterations","in","the","i-th","period","m_mul",":","A","scalar","`","float32","`","or","`","float64","`","`","Tensor","`","or","a","Python","number",".","Used","to","derive","the","initial","learning","rate","of","the","i-th","period",":","alpha",":","A","scalar","`","float32","`","or","`","float64","`","Tensor","or","a","Python","number",".","Minimum","learning","rate","value","as","a","fraction","of","the","initial_learning_rate",".","name",":","String",".","Optional","name","of","the","operation",".","Defaults","to","'SGDRDecay","'","."]
code: def __init__(
      self,
      initial_learning_rate,
      first_decay_steps,
      t_mul=2.0,
      m_mul=1.0,
      alpha=0.0,
      name=None,
  ):
      """Applies cosine decay with restarts to the learning rate.
  
      Args:
        initial_learning_rate: A scalar `float32` or `float64` Tensor or a Python
          number. The initial learning rate.
        first_decay_steps: A scalar `int32` or `int64` `Tensor` or a Python
          number. Number of steps to decay over.
        t_mul: A scalar `float32` or `float64` `Tensor` or a Python number.
          Used to derive the number of iterations in the i-th period
        m_mul: A scalar `float32` or `float64` `Tensor` or a Python number.
          Used to derive the initial learning rate of the i-th period:
        alpha: A scalar `float32` or `float64` Tensor or a Python number.
          Minimum learning rate value as a fraction of the initial_learning_rate.
        name: String. Optional name of the operation.  Defaults to 'SGDRDecay'.
      """
      super(CosineDecayRestarts, self).__init__()
  
      self.initial_learning_rate = initial_learning_rate
      self.first_decay_steps = first_decay_steps
      self._t_mul = t_mul
      self._m_mul = m_mul
      self.alpha = alpha
      self.name = name
  
imports: import abc
  import math
  from tensorflow.python.framework import constant_op
  from tensorflow.python.framework import ops
  from tensorflow.python.framework import tensor_conversion
  from tensorflow.python.keras.utils import generic_utils
  from tensorflow.python.ops import array_ops
  from tensorflow.python.ops import cond
  from tensorflow.python.ops import control_flow_case
  from tensorflow.python.ops import math_ops
  from tensorflow.python.ops import random_ops
  from tensorflow.python.util import nest
label_desc: Use image node_id 1 to create a new CosineDecayRestarts object from inherited base classes: LearningRateSchedule with example: obj = CosineDecayRestarts(initial_learning_rate, first_decay_steps, t_mul, m_mul, alpha, name)