class_docstr: Controller class for optimization.
  
  Mostly a thin wrapper for `optim`, but also useful for implementing learning rate
  scheduling beyond what is currently available. Also implements necessary methods for
  training RNNs such as grad manipulations.
  
  Args:
      method (string):
          One of [sgd, adagrad, adadelta, adam].
      lr (float):
          Learning rate.
      lr_decay (float):
          Learning rate decay multiplier.
      start_decay_at (int):
          Epoch to start learning rate decay. If None, starts decay when the validation
          accuracy stops improving. Defaults to 1.
      beta1, beta2 (float):
          Hyperarameters for adam.
      adagrad_accum (float, optional):
          Initialization hyperparameter for adagrad.
class_docstr_tok: ["Controller","class","for","optimization",".","Mostly","a","thin","wrapper","for","`","optim","`",",","but","also","useful","for","implementing","learning","rate","scheduling","beyond","what","is","currently","available",".","Also","implements","necessary","methods","for","training","RNNs","such","as","grad","manipulations",".","Args",":","method","(","string",")",":","One","of","[","sgd",",","adagrad",",","adadelta",",","adam","]",".","lr","(","float",")",":","Learning","rate",".","lr_decay","(","float",")",":","Learning","rate","decay","multiplier",".","start_decay_at","(","int",")",":","Epoch","to","start","learning","rate","decay",".","If","None",",","starts","decay","when","the","validation","accuracy","stops","improving",".","Defaults","to","1.","beta1",",","beta2","(","float",")",":","Hyperarameters","for","adam",".","adagrad_accum","(","float",",","optional",")",":","Initialization","hyperparameter","for","adagrad","."]
docstr: Decay learning rate.
  
  Decays lerning rate if val perf does not improve or we hit the `start_decay_at`
  limit.
  
  Args:
      acc:
          The accuracy score on the validation set.
      epoch:
          The current epoch number.
docstr_tok: ["Decay","learning","rate",".","Decays","lerning","rate","if","val","perf","does","not","improve","or","we","hit","the","`","start_decay_at","`","limit",".","Args",":","acc",":","The","accuracy","score","on","the","validation","set",".","epoch",":","The","current","epoch","number","."]
code: def update_learning_rate(self, acc, epoch):
      """Decay learning rate.
  
      Decays lerning rate if val perf does not improve or we hit the `start_decay_at`
      limit.
  
      Args:
          acc:
              The accuracy score on the validation set.
          epoch:
              The current epoch number.
      """
  
      if (
          self.start_decay_at is not None
          and epoch >= self.start_decay_at
      ):
          self.start_decay = True
      if self.last_acc is not None and acc < self.last_acc:
          self.start_decay = True
  
      if self.start_decay:
          self.lr = self.lr * self.lr_decay
          logger.info(
              "Setting learning rate to {:0.3e} for next epoch".format(
                  self.lr
              )
          )
  
      self.last_acc = acc
      self._set_rate(self.lr)
  
file_path: deepmatcher/deepmatcher/optim.py
imports: import logging
  import torch
  import torch.nn
  import torch.optim
  from torch.autograd import Variable
  from torch.nn.utils import clip_grad_norm_
label_desc: Use image node_id 5 for calling the Optimizer obj's underlying member method code with example usage: obj.update_learning_rate(acc, epoch) without return types