class_docstr: Since [`~optimization.Adafactor`] performs its own scheduling, if the training loop relies on a scheduler (e.g.,
  for logging), this class creates a proxy object that retrieves the current lr values from the optimizer.
  
  It returns `initial_lr` during startup and the actual `lr` during stepping.
class_docstr_tok: ["Since","[","`","~optimization.Adafactor","`","]","performs","its","own","scheduling",",","if","the","training","loop","relies","on","a","scheduler","(","e.g.",",","for","logging",")",",","this","class","creates","a","proxy","object","that","retrieves","the","current","lr","values","from","the","optimizer",".","It","returns","`","initial_lr","`","during","startup","and","the","actual","`","lr","`","during","stepping","."]
code: def __init__(self, optimizer, initial_lr=0.0):
      def lr_lambda(_):
          return initial_lr
  
      for group in optimizer.param_groups:
          group["initial_lr"] = initial_lr
      super().__init__(optimizer, lr_lambda)
      for group in optimizer.param_groups:
          del group["initial_lr"]
  
imports: import math
  import warnings
  from typing import Callable, Iterable, Optional, Tuple, Union
  import torch
  from torch import nn
  from torch.optim import Optimizer
  from torch.optim.lr_scheduler import LambdaLR
  from .trainer_utils import SchedulerType
  from .utils import logging
  from .utils.versions import require_version
label_desc: Use image node_id 1 to create a new AdafactorSchedule object from inherited base classes: LambdaLR with example: obj = AdafactorSchedule(optimizer, initial_lr)