class_docstr: Float16 optimizer wrapper for half precision (fp16 and bf16) data types.
  This optimizer wrapper holds main parameters and gradients in fp32 to support
  stable convergence.
  
  Arguments:
      optimizer: base optimizer such as Adam or SGD.
      fp32_grad_accum: to enable the use of fp32 in gradient accumulation and allreduce.
      contiguous_grad_bucket: to enable allocating the master gradients in the 
          contiguous memory space to reduce memory fragmentation.
      async_grad_allreduce: enable asynchronous gradient allreduce that is executed
          along with the training step backprop.
docstr: We only need to zero the model related parameters, i.e.,
  float16_groups & fp32_from_fp32_groups. We additionally zero
  fp32_from_float16_groups as a memory optimization to reduce
  fragmentation; in the case of set_to_none==True, the space
  used by this field can be safely deallocated at this point.
code: def zero_grad(self, set_to_none=True):
      """We only need to zero the model related parameters, i.e.,
      float16_groups & fp32_from_fp32_groups. We additionally zero
      fp32_from_float16_groups as a memory optimization to reduce
      fragmentation; in the case of set_to_none==True, the space
      used by this field can be safely deallocated at this point."""
      for group in self.float16_groups:
          _zero_grad_group_helper(group, set_to_none)
      if self._contiguous_grad_bucket:
          for i in self._main_grad_buffers:
              self._main_grad_buffers[i].zero()
      else:
          for group in self.fp32_from_float16_groups:
              _zero_grad_group_helper(group, set_to_none)
      for group in self.fp32_from_fp32_groups:
          _zero_grad_group_helper(group, set_to_none)
  
imports: from contextlib import contextmanager
  import torch
  from nemo.utils import logging
label_desc: Use image node_id 3 for calling the MainParamsOptimizerWrapper obj's underlying member method code with example usage: obj.zero_grad(set_to_none) without return types