class_docstr: Decay the LR by a factor every time the validation loss plateaus.
  Also comes with optional warmup phase, where we linearly increase
  the learning rate from some initial learning rate
  (``--warmup-init-lr``) until the configured learning rate
  (``--lr``). Thereafter the lr is adjusted according to original
  reduce_on_plateau scheme.
  
  During warmup::
  
    lrs = torch.linspace(
        cfg.warmup_init_lr, cfg.lr, cfg.warmup_updates
    )
    lr = lrs[update_num]
class_docstr_tok: ["Decay","the","LR","by","a","factor","every","time","the","validation","loss","plateaus",".","Also","comes","with","optional","warmup","phase",",","where","we","linearly","increase","the","learning","rate","from","some","initial","learning","rate","(","``","--","warmup-init-lr","``",")","until","the","configured","learning","rate","(","``","--","lr","``",")",".","Thereafter","the","lr","is","adjusted","according","to","original","reduce_on_plateau","scheme",".","During","warmup",":",":","lrs","=","torch.linspace","(","cfg.warmup_init_lr",",","cfg.lr",",","cfg.warmup_updates",")","lr","=","lrs","[","update_num","]"]
code: def __init__(self, cfg: ReduceLROnPlateauLRScheduleConfig, optimizer):
      super().__init__(cfg, optimizer)
      if len(cfg.lr) > 1:
          raise ValueError(
              "Cannot use a fixed learning rate schedule with reduce_lr_on_plateau."
              " Consider --lr-scheduler=fixed instead."
          )
      self.lr_scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(
          self.optimizer.optimizer,
          patience=cfg.lr_patience,
          factor=cfg.lr_shrink,
          mode="max" if cfg.maximize_best_checkpoint_metric else "min",
          threshold=cfg.lr_threshold,
      )
      warmup_end_lr = cfg.lr[0]
      # if no warm up, sets initial lr to be cfg.lr[0]
      if cfg.warmup_init_lr < 0:
          cfg.warmup_init_lr = (
              0 if cfg.warmup_updates > 0 else warmup_end_lr
          )
  
      # linearly warmup for the first cfg.warmup_updates
      if cfg.warmup_updates > 0:
          self.lr_step = (
              warmup_end_lr - cfg.warmup_init_lr
          ) / cfg.warmup_updates
  
      # this flag is either set from arg when no warm up, or set by
      # step_update() when warmup finishes
      self.warmup_end = True if cfg.warmup_updates <= 0 else False
  
      # initial learning rate
      # this self.lr is used only during init and/or warm up period
      self.lr = warmup_end_lr if self.warmup_end else cfg.warmup_init_lr
      self.optimizer.set_lr(self.lr)
  
file_path: fairseq/fairseq/optim/lr_scheduler/reduce_lr_on_plateau.py
imports: from dataclasses import dataclass, field
  from typing import List
  import torch.optim.lr_scheduler
  from omegaconf import II
  from fairseq.dataclass import FairseqDataclass
  from fairseq.optim.lr_scheduler import FairseqLRScheduler, register_lr_scheduler
label_desc: Use image node_id 1 to create a new ReduceLROnPlateauLRSchedule object from inherited base classes: FairseqLRScheduler with example: obj = ReduceLROnPlateauLRSchedule(cfg, optimizer)