class_docstr: Wrap an optimizer to perform memory-efficient mixed precision training.
  
  This class wraps an optimizer to perform FP16 training.
  This implementation is heavily based on the Fairseq implementation
  of `MemoryEfficientFP16Optimizer`, which can be found here:
  <https://github.com/pytorch/fairseq/blob/main/fairseq/optim/fp16_optimizer.py#L382>
  
  This allows you to train bigger models on a single GPU, but can be unstable.
  Prefer the SafeFP16 implementation if you do not have concerns about memory.
  
  :param params:
      Model parameters
  :param optimizer:
      Any torch optimizer
  :param float loss_initial_scale:
      Initial loss scaling. Default chosen empirically, but models with very low
      or high loss values may need this adjusted. Stick with powers of 2
  :param float min_loss_scale:
      Throws an error if your loss scale goes below this threshold
class_docstr_tok: ["Wrap","an","optimizer","to","perform","memory-efficient","mixed","precision","training",".","This","class","wraps","an","optimizer","to","perform","FP16","training",".","This","implementation","is","heavily","based","on","the","Fairseq","implementation","of","`","MemoryEfficientFP16Optimizer","`",",","which","can","be","found","here",":","<","https",":","\/\/github.com\/pytorch\/fairseq\/blob\/main\/fairseq\/optim\/fp16_optimizer.py","#","L382",">","This","allows","you","to","train","bigger","models","on","a","single","GPU",",","but","can","be","unstable",".","Prefer","the","SafeFP16","implementation","if","you","do","not","have","concerns","about","memory",".",":","param","params",":","Model","parameters",":","param","optimizer",":","Any","torch","optimizer",":","param","float","loss_initial_scale",":","Initial","loss","scaling",".","Default","chosen","empirically",",","but","models","with","very","low","or","high","loss","values","may","need","this","adjusted",".","Stick","with","powers","of","2",":","param","float","min_loss_scale",":","Throws","an","error","if","your","loss","scale","goes","below","this","threshold"]
docstr: Clears the gradients of all optimized parameters.
code: def zero_grad(self):
      """
      Clears the gradients of all optimized parameters.
      """
      self.optimizer.zero_grad()
      self._grads_are_scaled = False
  
imports: import math
  from itertools import chain
  from typing import Optional
  import parlai.utils.logging
label_desc: Use image node_id 18 for calling the MemoryEfficientFP16Optimizer obj's underlying member method code with example usage: obj.zero_grad() without return types