class_docstr: Float 16 Module.
  
  Attributes:
      config (TransformerConfig): Transformer config
      fp16 (bool) : Specifies if the model runs in fp16 mode
      bf16 (bool) : Specifies if the model runs in bf16 mode
  
  Args:
      config (TransformerConfig): The transformer config used to initalize the model
code: def __init__(
      self, config: TransformerConfig, module: torch.nn.Module
  ):
      super(Float16Module, self).__init__(config)
      self.config = config
      self.fp16 = config.fp16
      self.bf16 = config.bf16
  
      if self.fp16:
          self.add_module("module", module.half())
  
          def float16_convertor(val):
              return val.half()
  
      elif self.bf16:
          self.add_module("module", module.bfloat16())
  
          def float16_convertor(val):
              return val.bfloat16()
  
      else:
          raise Exception(
              "Either config.fp16 or config.bf16 should be True."
          )
  
      self.float16_convertor = float16_convertor
  
imports: import torch
  from torch.autograd import Variable
  from torch.nn.parameter import Parameter
  from megatron.core import parallel_state
  from megatron.core.transformer.transformer_config import TransformerConfig
label_desc: Use image node_id 1 to create a new Float16Module object from inherited base classes: MegatronModule with example: obj = Float16Module(config, module)