class_docstr: Sets the learning rate of each parameter group according to
  cyclical learning rate policy (CLR). The policy cycles the learning
  rate between two boundaries with a constant frequency, as detailed in
  the paper `Cyclical Learning Rates for Training Neural Networks`_.
  The distance between the two boundaries can be scaled on a per-iteration
  or per-cycle basis.
  
  Cyclical learning rate policy changes the learning rate after every batch.
  `batch_step` should be called after a batch has been used for training.
  To resume training, save `last_batch_iteration` and use it to instantiate `CycleLR`.
  
  This class has three built-in policies, as put forth in the paper:
  "triangular":
      A basic triangular cycle w/ no amplitude scaling.
  "triangular2":
      A basic triangular cycle that scales initial amplitude by half each cycle.
  "exp_range":
      A cycle that scales initial amplitude by gamma**(cycle iterations) at each
      cycle iteration.
  
  This implementation was adapted from the github repo: `bckenstler/CLR`_
  
  :param optimizer: (Optimizer):
      Wrapped optimizer.
  :param base_lr: (float or list):
      Initial learning rate which is the
      lower boundary in the cycle for eachparam groups.
      Default: 0.001
  :param max_lr: (float or list): Upper boundaries in the cycle for
      each parameter group. Functionally,
      it defines the cycle amplitude (max_lr - base_lr).
      The lr at any cycle is the sum of base_lr
      and some scaling of the amplitude; therefore
      max_lr may not actually be reached depending on
      scaling function. Default: 0.006
  :param step_size: (int): Number of training iterations per
      half cycle. Authors suggest setting step_size
      2-8 x training iterations in epoch. Default: 2000
  :param mode: (str): One of {triangular, triangular2, exp_range}.
      Values correspond to policies detailed above.
      If scale_fn is not None, this argument is ignored.
      Default: 'triangular'
  :param gamma: (float): Constant in 'exp_range' scaling function:
      gamma**(cycle iterations)
      Default: 1.0
  :param scale_fn: (function): Custom scaling policy defined by a single
      argument lambda function, where
      0 <= scale_fn(x) <= 1 for all x >= 0.
      mode paramater is ignored
      Default: None
  :param scale_mode: (str): {'cycle', 'iterations'}.
      Defines whether scale_fn is evaluated on
      cycle number or cycle iterations (training
      iterations since start of cycle).
      Default: 'cycle'
  :param verbose: (bool): Whether to produce some output during initialization
      Default: True
  
  Example:
      >>> optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
      >>> scheduler = torch.optim.CyclicLR(optimizer)
      >>> data_loader = torch.utils.data.DataLoader(...)
      >>> for epoch in range(10):
      >>>     for batch in data_loader:
      >>>         scheduler.batch_step()
      >>>         train_batch(...)
  
  .. _Cyclical Learning Rates for Training Neural Networks: https://arxiv.org/abs/1506.01186
  .. _bckenstler/CLR: https://github.com/bckenstler/CLR
class_docstr_tok: ["Sets","the","learning","rate","of","each","parameter","group","according","to","cyclical","learning","rate","policy","(","CLR",")",".","The","policy","cycles","the","learning","rate","between","two","boundaries","with","a","constant","frequency",",","as","detailed","in","the","paper","`","Cyclical","Learning","Rates","for","Training","Neural","Networks","`","_",".","The","distance","between","the","two","boundaries","can","be","scaled","on","a","per-iteration","or","per-cycle","basis",".","Cyclical","learning","rate","policy","changes","the","learning","rate","after","every","batch",".","`","batch_step","`","should","be","called","after","a","batch","has","been","used","for","training",".","To","resume","training",",","save","`","last_batch_iteration","`","and","use","it","to","instantiate","`","CycleLR","`",".","This","class","has","three","built-in","policies",",","as","put","forth","in","the","paper",":","''","triangular","''",":","A","basic","triangular","cycle","w\/","no","amplitude","scaling",".","``","triangular2","''",":","A","basic","triangular","cycle","that","scales","initial","amplitude","by","half","each","cycle",".","``","exp_range","''",":","A","cycle","that","scales","initial","amplitude","by","gamma","*","*","(","cycle","iterations",")","at","each","cycle","iteration",".","This","implementation","was","adapted","from","the","github","repo",":","`","bckenstler\/CLR","`","_",":","param","optimizer",":","(","Optimizer",")",":","Wrapped","optimizer",".",":","param","base_lr",":","(","float","or","list",")",":","Initial","learning","rate","which","is","the","lower","boundary","in","the","cycle","for","eachparam","groups",".","Default",":","0.001",":","param","max_lr",":","(","float","or","list",")",":","Upper","boundaries","in","the","cycle","for","each","parameter","group",".","Functionally",",","it","defines","the","cycle","amplitude","(","max_lr","-","base_lr",")",".","The","lr","at","any","cycle","is","the","sum","of","base_lr","and","some","scaling","of","the","amplitude",";","therefore","max_lr","may","not","actually","be","reached","depending","on","scaling","function",".","Default",":","0.006",":","param","step_size",":","(","int",")",":","Number","of","training","iterations","per","half","cycle",".","Authors","suggest","setting","step_size","2-8","x","training","iterations","in","epoch",".","Default",":","2000",":","param","mode",":","(","str",")",":","One","of","{","triangular",",","triangular2",",","exp_range","}",".","Values","correspond","to","policies","detailed","above",".","If","scale_fn","is","not","None",",","this","argument","is","ignored",".","Default",":","'triangular'",":","param","gamma",":","(","float",")",":","Constant","in","'exp_range","'","scaling","function",":","gamma","*","*","(","cycle","iterations",")","Default",":","1.0",":","param","scale_fn",":","(","function",")",":","Custom","scaling","policy","defined","by","a","single","argument","lambda","function",",","where","0","<","=","scale_fn","(","x",")","<","=","1","for","all","x",">","=","0.","mode","paramater","is","ignored","Default",":","None",":","param","scale_mode",":","(","str",")",":","{","'cycle","'",",","'iterations","'","}",".","Defines","whether","scale_fn","is","evaluated","on","cycle","number","or","cycle","iterations","(","training","iterations","since","start","of","cycle",")",".","Default",":","'cycle'",":","param","verbose",":","(","bool",")",":","Whether","to","produce","some","output","during","initialization","Default",":","True","Example",":",">",">",">","optimizer","=","torch.optim.SGD","(","model.parameters","(",")",",","lr=0.1",",","momentum=0.9",")",">",">",">","scheduler","=","torch.optim.CyclicLR","(","optimizer",")",">",">",">","data_loader","=","torch.utils.data.DataLoader","(","...",")",">",">",">","for","epoch","in","range","(","10",")",":",">",">",">","for","batch","in","data_loader",":",">",">",">","scheduler.batch_step","(",")",">",">",">","train_batch","(","...",")","..","_Cyclical","Learning","Rates","for","Training","Neural","Networks",":","https",":","\/\/arxiv.org\/abs\/1506.01186","..","_bckenstler\/CLR",":","https",":","\/\/github.com\/bckenstler\/CLR"]
code: def _exp_range_scale_fn(self, x):
      return self.gamma ** (x)
  
file_path: pywick/pywick/callbacks/CyclicLRScheduler.py
imports: from torch.optim.optimizer import Optimizer
  import numpy
  from .None import Callback
  from ..misc import trun_n_d
label_desc: Use image node_id 5 for calling the CyclicLRScheduler obj's underlying member method code with example usage: obj._exp_range_scale_fn(x) and returns: unknown