class_docstr: Float16 optimizer wrapper for half precision (fp16 and bf16) data types.
  This optimizer wrapper holds main parameters and gradients in fp32 to support
  stable convergence.
  
  Arguments:
      optimizer: base optimizer such as Adam or SGD.
      fp32_grad_accum: to enable the use of fp32 in gradient accumulation and allreduce.
      contiguous_grad_bucket: to enable allocating the master gradients in the 
          contiguous memory space to reduce memory fragmentation.
      async_grad_allreduce: enable asynchronous gradient allreduce that is executed
          along with the training step backprop.
code: def load_state_dict(self, state_dict):
      # Optimizer.
      optimizer_key = "optimizer"
      if optimizer_key not in state_dict:
          optimizer_key = "optimizer_state_dict"
          logging.info(
              "***WARNING*** loading optimizer from "
              "an old checkpoint ..."
          )
      self.optimizer.load_state_dict(state_dict[optimizer_key])
  
      # Copy data for the main params.
      fp32_from_float16_params_key = "fp32_from_fp16_params"
      if fp32_from_float16_params_key not in state_dict:
          fp32_from_float16_params_key = "fp32_from_fp16"
      for current_group, saved_group in zip(
          self.fp32_from_float16_groups,
          state_dict[fp32_from_float16_params_key],
      ):
          for current_param, saved_param in zip(
              current_group, saved_group
          ):
              current_param.data.copy_(saved_param.data)
  
imports: from contextlib import contextmanager
  import torch
  from nemo.utils import logging
label_desc: Use image node_id 12 for calling the MainParamsOptimizerWrapper obj's underlying member method code with example usage: obj.load_state_dict(state_dict) without return types