class_docstr: Controller class for optimization.
  
  Mostly a thin wrapper for `optim`, but also useful for implementing learning rate
  scheduling beyond what is currently available. Also implements necessary methods for
  training RNNs such as grad manipulations.
  
  Args:
      method (string):
          One of [sgd, adagrad, adadelta, adam].
      lr (float):
          Learning rate.
      lr_decay (float):
          Learning rate decay multiplier.
      start_decay_at (int):
          Epoch to start learning rate decay. If None, starts decay when the validation
          accuracy stops improving. Defaults to 1.
      beta1, beta2 (float):
          Hyperarameters for adam.
      adagrad_accum (float, optional):
          Initialization hyperparameter for adagrad.
class_docstr_tok: ["Controller","class","for","optimization",".","Mostly","a","thin","wrapper","for","`","optim","`",",","but","also","useful","for","implementing","learning","rate","scheduling","beyond","what","is","currently","available",".","Also","implements","necessary","methods","for","training","RNNs","such","as","grad","manipulations",".","Args",":","method","(","string",")",":","One","of","[","sgd",",","adagrad",",","adadelta",",","adam","]",".","lr","(","float",")",":","Learning","rate",".","lr_decay","(","float",")",":","Learning","rate","decay","multiplier",".","start_decay_at","(","int",")",":","Epoch","to","start","learning","rate","decay",".","If","None",",","starts","decay","when","the","validation","accuracy","stops","improving",".","Defaults","to","1.","beta1",",","beta2","(","float",")",":","Hyperarameters","for","adam",".","adagrad_accum","(","float",",","optional",")",":","Initialization","hyperparameter","for","adagrad","."]
code: def __init__(
      self,
      method="adam",
      lr=0.001,
      max_grad_norm=5,
      start_decay_at=1,
      beta1=0.9,
      beta2=0.999,
      adagrad_accum=0.0,
      lr_decay=0.8,
  ):
      self.last_acc = None
      self.lr = lr
      self.original_lr = lr
      self.max_grad_norm = max_grad_norm
      self.method = method
      self.lr_decay = lr_decay
      self.start_decay_at = start_decay_at
      self.start_decay = False
      self._step = 0
      self.betas = [beta1, beta2]
      self.adagrad_accum = adagrad_accum
      self.params = None
      logger.info("Initial learning rate: {:0.3e}".format(self.lr))
  
file_path: deepmatcher/deepmatcher/optim.py
imports: import logging
  import torch
  import torch.nn
  import torch.optim
  from torch.autograd import Variable
  from torch.nn.utils import clip_grad_norm_
label_desc: Use image node_id 1 to create a new Optimizer object from inherited base classes: object with example: obj = Optimizer(method, lr, max_grad_norm, start_decay_at, beta1, beta2, adagrad_accum, lr_decay)