class_docstr: Reduce learning rate and weight decay when a metric has stopped improving.
  
  Models often benefit from reducing the learning rate by
  a factor of 2-10 once learning stagnates. This scheduler reads a metric
  quantity and if no improvement is seen for a 'patience' number
  of epochs, the learning rate and weight decay factor is reduced for
  optimizers that implement the the weight decay method from the paper
  `Fixing Weight Decay Regularization in Adam`_.
  
  .. _Fixing Weight Decay Regularization in Adam:
      https://arxiv.org/abs/1711.05101
  
  Args:
  ----
      optimizer (Optimizer): Wrapped optimizer.
      mode (str): One of `min`, `max`. In `min` mode, lr will
          be reduced when the quantity monitored has stopped
          decreasing; in `max` mode it will be reduced when the
          quantity monitored has stopped increasing. Default: 'min'.
      factor (float): Factor by which the learning rate will be
          reduced. new_lr = lr * factor. Default: 0.1.
      patience (int): Number of epochs with no improvement after
          which learning rate will be reduced. For example, if
          `patience = 2`, then we will ignore the first 2 epochs
          with no improvement, and will only decrease the LR after the
          3rd epoch if the loss still hasn't improved then.
          Default: 10.
      verbose (bool): If ``True``, prints a message to stdout for
          each update. Default: ``False``.
      threshold (float): Threshold for measuring the new optimum,
          to only focus on significant changes. Default: 1e-4.
      threshold_mode (str): One of `rel`, `abs`. In `rel` mode,
          dynamic_threshold = best * ( 1 + threshold ) in 'max'
          mode or best * ( 1 - threshold ) in `min` mode.
          In `abs` mode, dynamic_threshold = best + threshold in
          `max` mode or best - threshold in `min` mode. Default: 'rel'.
      cooldown (int): Number of epochs to wait before resuming
          normal operation after lr has been reduced. Default: 0.
      min_lr (float or list): A scalar or a list of scalars. A
          lower bound on the learning rate of all param groups
          or each group respectively. Default: 0.
      eps (float): Minimal decay applied to lr. If the difference
          between new and old lr is smaller than eps, the update is
          ignored. Default: 1e-8.
  
  Example:
  -------
      >>> optimizer = AdamW(model.parameters(), lr=0.1, weight_decay=1e-3)
      >>> scheduler = ReduceLRWDOnPlateau(optimizer, 'min')
      >>> for epoch in range(10):
      >>>     train(...)
      >>>     val_loss = validate(...)
      >>>     # Note that step should be called after validate()
      >>>     scheduler.step(val_loss)
class_docstr_tok: ["Reduce","learning","rate","and","weight","decay","when","a","metric","has","stopped","improving",".","Models","often","benefit","from","reducing","the","learning","rate","by","a","factor","of","2-10","once","learning","stagnates",".","This","scheduler","reads","a","metric","quantity","and","if","no","improvement","is","seen","for","a","'patience","'","number","of","epochs",",","the","learning","rate","and","weight","decay","factor","is","reduced","for","optimizers","that","implement","the","the","weight","decay","method","from","the","paper","`","Fixing","Weight","Decay","Regularization","in","Adam","`","_",".","..","_Fixing","Weight","Decay","Regularization","in","Adam",":","https",":","\/\/arxiv.org\/abs\/1711.05101","Args",":","--","--","optimizer","(","Optimizer",")",":","Wrapped","optimizer",".","mode","(","str",")",":","One","of","`","min","`",",","`","max","`",".","In","`","min","`","mode",",","lr","will","be","reduced","when","the","quantity","monitored","has","stopped","decreasing",";","in","`","max","`","mode","it","will","be","reduced","when","the","quantity","monitored","has","stopped","increasing",".","Default",":","'min","'",".","factor","(","float",")",":","Factor","by","which","the","learning","rate","will","be","reduced",".","new_lr","=","lr","*","factor",".","Default",":","0.1.","patience","(","int",")",":","Number","of","epochs","with","no","improvement","after","which","learning","rate","will","be","reduced",".","For","example",",","if","`","patience","=","2","`",",","then","we","will","ignore","the","first","2","epochs","with","no","improvement",",","and","will","only","decrease","the","LR","after","the","3rd","epoch","if","the","loss","still","has","n't","improved","then",".","Default",":","10.","verbose","(","bool",")",":","If","``","True","``",",","prints","a","message","to","stdout","for","each","update",".","Default",":","``","False","``",".","threshold","(","float",")",":","Threshold","for","measuring","the","new","optimum",",","to","only","focus","on","significant","changes",".","Default",":","1e-4",".","threshold_mode","(","str",")",":","One","of","`","rel","`",",","`","abs","`",".","In","`","rel","`","mode",",","dynamic_threshold","=","best","*","(","1","+","threshold",")","in","'max'","mode","or","best","*","(","1","-","threshold",")","in","`","min","`","mode",".","In","`","abs","`","mode",",","dynamic_threshold","=","best","+","threshold","in","`","max","`","mode","or","best","-","threshold","in","`","min","`","mode",".","Default",":","'rel","'",".","cooldown","(","int",")",":","Number","of","epochs","to","wait","before","resuming","normal","operation","after","lr","has","been","reduced",".","Default",":","0.","min_lr","(","float","or","list",")",":","A","scalar","or","a","list","of","scalars",".","A","lower","bound","on","the","learning","rate","of","all","param","groups","or","each","group","respectively",".","Default",":","0.","eps","(","float",")",":","Minimal","decay","applied","to","lr",".","If","the","difference","between","new","and","old","lr","is","smaller","than","eps",",","the","update","is","ignored",".","Default",":","1e-8",".","Example",":","--","--","--","-",">",">",">","optimizer","=","AdamW","(","model.parameters","(",")",",","lr=0.1",",","weight_decay=1e-3",")",">",">",">","scheduler","=","ReduceLRWDOnPlateau","(","optimizer",",","'min","'",")",">",">",">","for","epoch","in","range","(","10",")",":",">",">",">","train","(","...",")",">",">",">","val_loss","=","validate","(","...",")",">",">",">","#","Note","that","step","should","be","called","after","validate","(",")",">",">",">","scheduler.step","(","val_loss",")"]
code: def _reduce_weight_decay(self, epoch):
      for i, param_group in enumerate(self.optimizer.param_groups):
          if param_group["weight_decay"] != 0:
              old_weight_decay = float(param_group["weight_decay"])
              new_weight_decay = max(
                  old_weight_decay * self.factor, self.min_lrs[i]
              )
              if old_weight_decay - new_weight_decay > self.eps:
                  param_group["weight_decay"] = new_weight_decay
                  if self.verbose:
                      log.info(
                          f"Epoch {epoch}: reducing weight decay factor of group {i} to {new_weight_decay:.4e}."
                      )
  
imports: import logging
  import torch
  from torch.optim import Optimizer
  from torch.optim.lr_scheduler import LambdaLR, ReduceLROnPlateau, _LRScheduler
  from torch.optim.optimizer import required
label_desc: Use image node_id 2 for calling the ReduceLRWDOnPlateau obj's underlying member method code with example usage: obj._reduce_weight_decay(epoch) without return types