class_docstr: Scheduler that decays at an inverse square root rate.
docstr: invsqrt_lr_decay_gamma determines the cycle length of the inverse square root
  scheduler.
  
  When steps taken == invsqrt_lr_decay_gamma, the lr multiplier is 1
docstr_tok: ["invsqrt_lr_decay_gamma","determines","the","cycle","length","of","the","inverse","square","root","scheduler",".","When","steps","taken","==","invsqrt_lr_decay_gamma",",","the","lr","multiplier","is","1"]
code: def __init__(
      self,
      optimizer,
      patience,
      decay,
      warmup_updates,
      warmup_rate,
      invsqrt_lr_decay_gamma,
      max_lr_steps,
  ):
      """
      invsqrt_lr_decay_gamma determines the cycle length of the inverse square root
      scheduler.
  
      When steps taken == invsqrt_lr_decay_gamma, the lr multiplier is 1
      """
      super().__init__(warmup_updates, warmup_rate)
      assert self.warmup_updates >= 0
      self.max_lr_steps = max_lr_steps - self.warmup_updates
      self.invsqrt_lr_decay_gamma = invsqrt_lr_decay_gamma
      if invsqrt_lr_decay_gamma <= 0:
          warn_once(
              "--lr-scheduler invsqrt requires a value for "
              "--invsqrt-lr-decay-gamma. Defaulting to set gamma to "
              "--warmup-updates value for backwards compatibility."
          )
          self.invsqrt_lr_decay_gamma = self.warmup_updates
  
      self.decay_factor = np.sqrt(max(1, self.invsqrt_lr_decay_gamma))
      self.scheduler = optim.lr_scheduler.LambdaLR(
          optimizer, self._invsqrt_lr
      )
  
imports: import math
  from typing import Optional
  from parlai.core.params import ParlaiParser
  from parlai.core.opt import Opt
  from abc import abstractmethod
  from torch import optim
  import numpy
  from parlai.core.exceptions import StopTrainException
  from parlai.utils.misc import warn_once
label_desc: Use image node_id 1 to create a new InvSqrtLRScheduler object from inherited base classes: ParlAILRScheduler with example: obj = InvSqrtLRScheduler(optimizer, patience, decay, warmup_updates, warmup_rate, invsqrt_lr_decay_gamma, max_lr_steps)