class_docstr: Controller class for optimization. Mostly a thin
  wrapper for `optim`, but also useful for implementing
  rate scheduling beyond what is currently available.
  Also implements necessary methods for training RNNs such
  as grad manipulations.
  
  Args:
      optimizer: A ``torch.optim.Optimizer`` instance.
      learning_rate: The initial learning rate.
      learning_rate_decay_fn: An optional callable taking the current step
          as argument and return a learning rate scaling factor.
      max_grad_norm: Clip gradients to this global norm.
class_docstr_tok: ["Controller","class","for","optimization",".","Mostly","a","thin","wrapper","for","`","optim","`",",","but","also","useful","for","implementing","rate","scheduling","beyond","what","is","currently","available",".","Also","implements","necessary","methods","for","training","RNNs","such","as","grad","manipulations",".","Args",":","optimizer",":","A","``","torch.optim.Optimizer","``","instance",".","learning_rate",":","The","initial","learning","rate",".","learning_rate_decay_fn",":","An","optional","callable","taking","the","current","step","as","argument","and","return","a","learning","rate","scaling","factor",".","max_grad_norm",":","Clip","gradients","to","this","global","norm","."]
docstr: Wrapper for backward pass. Some optimizer requires ownership of the
  backward pass.
docstr_tok: ["Wrapper","for","backward","pass",".","Some","optimizer","requires","ownership","of","the","backward","pass","."]
code: def backward(self, loss):
      """Wrapper for backward pass. Some optimizer requires ownership of the
      backward pass."""
      if self._fp16 == "legacy":
          kwargs = {}
          if "update_master_grads" in fn_args(self._optimizer.backward):
              kwargs["update_master_grads"] = True
          self._optimizer.backward(loss, **kwargs)
      elif self.amp:
          self._scaler.scale(loss).backward()
      elif self._fp16 == "apex.amp":
          with apex.amp.scale_loss(
              loss, self._optimizer
          ) as scaled_loss:
              scaled_loss.backward()
      else:
          loss.backward()
  
imports: import torch
  import torch.optim
  from torch.nn.utils import clip_grad_norm_
  import operator
  import functools
  from copy import copy
  from math import sqrt
  import types
  import os
  import importlib
  from onmt.utils.misc import fn_args
label_desc: Use image node_id 9 for calling the Optimizer obj's underlying member method code with example usage: obj.backward(loss) without return types