class_docstr: Float16 optimizer wrapper for half precision (fp16 and bf16) data types.
  This optimizer wrapper holds main parameters and gradients in fp32 to support
  stable convergence.
  
  Arguments:
      optimizer: base optimizer such as Adam or SGD.
      fp32_grad_accum: to enable the use of fp32 in gradient accumulation and allreduce.
      contiguous_grad_bucket: to enable allocating the master gradients in the 
          contiguous memory space to reduce memory fragmentation.
      async_grad_allreduce: enable asynchronous gradient allreduce that is executed
          along with the training step backprop.
docstr: Initialize the optimizer state with zero tensors for 'exp_avg' and 'exp_avg_sq' of each parameter.
code: def _init_opt_state(self):
      """
      Initialize the optimizer state with zero tensors for 'exp_avg' and 'exp_avg_sq' of each parameter.
      """
      for group in self.optimizer.param_groups:
          for p in group["params"]:
              if len(self.optimizer.state[p]) == 0:
                  self.optimizer.state[p]["exp_avg"] = torch.zeros_like(
                      p.data
                  )
                  self.optimizer.state[p][
                      "exp_avg_sq"
                  ] = torch.zeros_like(p.data)
  
imports: from contextlib import contextmanager
  import torch
  from nemo.utils import logging
label_desc: Use image node_id 24 for calling the MainParamsOptimizerWrapper obj's underlying member method code with example usage: obj._init_opt_state() without return types