class_docstr: Sets the learning rate of each parameter group according to the 1cycle learning rate policy. The 1cycle policy anneals
  the learning rate from an initial learning rate to some maximum learning rate and then from that maximum learning rate
  to some minimum learning rate much lower than the initial learning rate. This policy was initially described in the
  paper Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates.
  
  :param optimizer: (Optimizer)           : - Wrapped optimizer.
  :param max_lr (float or list)           : – Upper learning rate boundaries in the cycle for each parameter group.
  :param total_steps (int)                : – The total number of steps in the cycle. Note that if a value is not provided here, then it must be inferred by providing a value for epochs and steps_per_epoch. Default: None
  :param epochs (int)                     : – The number of epochs to train for. This is used along with steps_per_epoch in order to infer the total number of steps in the cycle if a value for total_steps is not provided. Default: None
  :param steps_per_epoch (int)            : – The number of steps per epoch to train for. This is used along with epochs in order to infer the total number of steps in the cycle if a value for total_steps is not provided. Default: None
  :param pct_start (float)                : – The percentage of the cycle (in number of steps) spent increasing the learning rate. Default: 0.3
  :param anneal_strategy (str)            : – {‘cos’, ‘linear’} Specifies the annealing strategy: “cos” for cosine annealing, “linear” for linear annealing. Default: ‘cos’
  :param cycle_momentum (bool)            : – If True, momentum is cycled inversely to learning rate between ‘base_momentum’ and ‘max_momentum’. Default: True
  :param base_momentum (float or list)    : – Lower momentum boundaries in the cycle for each parameter group. Note that momentum is cycled inversely to learning rate; at the peak of a cycle, momentum is ‘base_momentum’ and learning rate is ‘max_lr’. Default: 0.85
  :param max_momentum (float or list)     : – Upper momentum boundaries in the cycle for each parameter group. Functionally, it defines the cycle amplitude (max_momentum - base_momentum). Note that momentum is cycled inversely to learning rate; at the start of a cycle, momentum is ‘max_momentum’ and learning rate is ‘base_lr’ Default: 0.95
  :param div_factor (float)               : – Determines the initial learning rate via initial_lr = max_lr/div_factor Default: 25
  :param final_div_factor (float)         : – Determines the minimum learning rate via min_lr = initial_lr/final_div_factor Default: 1e4
  :param last_epoch (int)                 : – The index of the last batch. This parameter is used when resuming a training job. Since step() should be invoked after each batch instead of after each epoch, this number represents the total number of batches computed, not the total number of epochs computed. When last_epoch=-1, the schedule is started from the beginning. Default: -1
  
  Example:
      >>> import torch
      >>> data_loader = torch.utils.data.DataLoader(...)
      >>> optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
      >>> scheduler = torch.optim.lr_scheduler.OneCycleLR(optimizer, max_lr=0.01, steps_per_epoch=len(data_loader), epochs=10)
      >>> for epoch in range(10):
      >>>     for batch in data_loader:
      >>>         train_batch(...)
      >>>         scheduler.step()
class_docstr_tok: ["Sets","the","learning","rate","of","each","parameter","group","according","to","the","1cycle","learning","rate","policy",".","The","1cycle","policy","anneals","the","learning","rate","from","an","initial","learning","rate","to","some","maximum","learning","rate","and","then","from","that","maximum","learning","rate","to","some","minimum","learning","rate","much","lower","than","the","initial","learning","rate",".","This","policy","was","initially","described","in","the","paper","Super-Convergence",":","Very","Fast","Training","of","Neural","Networks","Using","Large","Learning","Rates",".",":","param","optimizer",":","(","Optimizer",")",":","-","Wrapped","optimizer",".",":","param","max_lr","(","float","or","list",")",":","\u2013","Upper","learning","rate","boundaries","in","the","cycle","for","each","parameter","group",".",":","param","total_steps","(","int",")",":","\u2013","The","total","number","of","steps","in","the","cycle",".","Note","that","if","a","value","is","not","provided","here",",","then","it","must","be","inferred","by","providing","a","value","for","epochs","and","steps_per_epoch",".","Default",":","None",":","param","epochs","(","int",")",":","\u2013","The","number","of","epochs","to","train","for",".","This","is","used","along","with","steps_per_epoch","in","order","to","infer","the","total","number","of","steps","in","the","cycle","if","a","value","for","total_steps","is","not","provided",".","Default",":","None",":","param","steps_per_epoch","(","int",")",":","\u2013","The","number","of","steps","per","epoch","to","train","for",".","This","is","used","along","with","epochs","in","order","to","infer","the","total","number","of","steps","in","the","cycle","if","a","value","for","total_steps","is","not","provided",".","Default",":","None",":","param","pct_start","(","float",")",":","\u2013","The","percentage","of","the","cycle","(","in","number","of","steps",")","spent","increasing","the","learning","rate",".","Default",":","0.3",":","param","anneal_strategy","(","str",")",":","\u2013","{","\u2018","cos","\u2019",",","\u2018","linear","\u2019","}","Specifies","the","annealing","strategy",":","\u201c","cos","\u201d","for","cosine","annealing",",","\u201c","linear","\u201d","for","linear","annealing",".","Default",":","\u2018","cos","\u2019",":","param","cycle_momentum","(","bool",")",":","\u2013","If","True",",","momentum","is","cycled","inversely","to","learning","rate","between","\u2018","base_momentum","\u2019","and","\u2018","max_momentum","\u2019",".","Default",":","True",":","param","base_momentum","(","float","or","list",")",":","\u2013","Lower","momentum","boundaries","in","the","cycle","for","each","parameter","group",".","Note","that","momentum","is","cycled","inversely","to","learning","rate",";","at","the","peak","of","a","cycle",",","momentum","is","\u2018","base_momentum","\u2019","and","learning","rate","is","\u2018","max_lr","\u2019",".","Default",":","0.85",":","param","max_momentum","(","float","or","list",")",":","\u2013","Upper","momentum","boundaries","in","the","cycle","for","each","parameter","group",".","Functionally",",","it","defines","the","cycle","amplitude","(","max_momentum","-","base_momentum",")",".","Note","that","momentum","is","cycled","inversely","to","learning","rate",";","at","the","start","of","a","cycle",",","momentum","is","\u2018","max_momentum","\u2019","and","learning","rate","is","\u2018","base_lr","\u2019","Default",":","0.95",":","param","div_factor","(","float",")",":","\u2013","Determines","the","initial","learning","rate","via","initial_lr","=","max_lr\/div_factor","Default",":","25",":","param","final_div_factor","(","float",")",":","\u2013","Determines","the","minimum","learning","rate","via","min_lr","=","initial_lr\/final_div_factor","Default",":","1e4",":","param","last_epoch","(","int",")",":","\u2013","The","index","of","the","last","batch",".","This","parameter","is","used","when","resuming","a","training","job",".","Since","step","(",")","should","be","invoked","after","each","batch","instead","of","after","each","epoch",",","this","number","represents","the","total","number","of","batches","computed",",","not","the","total","number","of","epochs","computed",".","When","last_epoch=-1",",","the","schedule","is","started","from","the","beginning",".","Default",":","-1","Example",":",">",">",">","import","torch",">",">",">","data_loader","=","torch.utils.data.DataLoader","(","...",")",">",">",">","optimizer","=","torch.optim.SGD","(","model.parameters","(",")",",","lr=0.1",",","momentum=0.9",")",">",">",">","scheduler","=","torch.optim.lr_scheduler.OneCycleLR","(","optimizer",",","max_lr=0.01",",","steps_per_epoch=len","(","data_loader",")",",","epochs=10",")",">",">",">","for","epoch","in","range","(","10",")",":",">",">",">","for","batch","in","data_loader",":",">",">",">","train_batch","(","...",")",">",">",">","scheduler.step","(",")"]
code: def __init__(
      self,
      optimizer,
      max_lr,
      total_steps=None,
      epochs=None,
      steps_per_epoch=None,
      pct_start=0.3,
      anneal_strategy="cos",
      cycle_momentum=True,
      base_momentum=0.85,
      max_momentum=0.95,
      div_factor=25.0,
      final_div_factor=10000.0,
      last_epoch=-1,
      verbose=True,
  ):
      self.scheduler = OneCycleLR(
          optimizer=optimizer,
          max_lr=max_lr,
          total_steps=total_steps,
          epochs=epochs,
          steps_per_epoch=steps_per_epoch,
          pct_start=pct_start,
          anneal_strategy=anneal_strategy,
          cycle_momentum=cycle_momentum,
          base_momentum=base_momentum,
          max_momentum=max_momentum,
          div_factor=div_factor,
          final_div_factor=final_div_factor,
          last_epoch=last_epoch,
      )
  
      if verbose:
          print("OneCycleLRScheduler params:")
          print("\tmax_lr: {}".format(max_lr))
          print("\tepochs: {}".format(epochs))
          print("\tsteps_per_epoch: {}".format(steps_per_epoch))
          print("\tpct_start: {}".format(pct_start))
          print("\tdiv_factor: {}".format(div_factor))
          print("\tfinal_div_factor: {}".format(final_div_factor))
  
      self.optimizer_name = optimizer.__class__.__name__.lower()
  
file_path: pywick/pywick/callbacks/OneCycleLRScheduler.py
imports: import math
  import warnings
  from torch.optim import Optimizer
  from .None import Callback
  import torch
label_desc: Use image node_id 1 to create a new OnceCycleLRScheduler object from inherited base classes: Callback with example: obj = OnceCycleLRScheduler(optimizer, max_lr, total_steps, epochs, steps_per_epoch, pct_start, anneal_strategy, cycle_momentum, base_momentum, max_momentum, div_factor, final_div_factor, last_epoch, verbose)