class_docstr: Float16 optimizer wrapper for half precision (fp16 and bf16) data types.
  This optimizer wrapper holds main parameters and gradients in fp32 to support
  stable convergence.
  
  Arguments:
      optimizer: base optimizer such as Adam or SGD.
      fp32_grad_accum: to enable the use of fp32 in gradient accumulation and allreduce.
      contiguous_grad_bucket: to enable allocating the master gradients in the 
          contiguous memory space to reduce memory fragmentation.
      async_grad_allreduce: enable asynchronous gradient allreduce that is executed
          along with the training step backprop.
code: def _copy_model_params_to_main_params(self):
      # Only needed for the float16 params.
      (
          model_data,
          main_data,
          half_dtype,
      ) = self._get_model_and_main_params_data_float16()
      self._set_overflow_buffer(half_dtype)
      _multi_tensor_copy_this_to_that(
          this=model_data,
          that=main_data,
          overflow_buf=self._dummy_overflow_buf,
      )
  
imports: from contextlib import contextmanager
  import torch
  from nemo.utils import logging
label_desc: Use image node_id 8 for calling the MainParamsOptimizerWrapper obj's underlying member method code with example usage: obj._copy_model_params_to_main_params() without return types