class_docstr: Masked LM head for Bert
  
  Arguments:
      mpu_vocab_size: model parallel size of vocabulary.
      hidden_size: hidden size
      init_method: init method for weight initialization
      layernorm_epsilon: tolerance for layer norm divisions
      parallel_output: whether output logits being distributed or not.
class_docstr_tok: ["Masked","LM","head","for","Bert","Arguments",":","mpu_vocab_size",":","model","parallel","size","of","vocabulary",".","hidden_size",":","hidden","size","init_method",":","init","method","for","weight","initialization","layernorm_epsilon",":","tolerance","for","layer","norm","divisions","parallel_output",":","whether","output","logits","being","distributed","or","not","."]
code: def __init__(
      self,
      config: ModelParallelConfig,
      mpu_vocab_size,
      hidden_size,
      init_method,
      layernorm_epsilon,
      parallel_output,
      use_openai_gelu,
      onnx_safe,
  ):
      super(BertLMHead, self).__init__(config=config)
  
      self.bias = torch.nn.Parameter(torch.zeros(mpu_vocab_size))
      set_tensor_model_parallel_attributes(self.bias, True, 0, 1)
      self.parallel_output = parallel_output
      self.sequence_parallel = config.sequence_parallel
  
      self.dense = get_linear_layer(
          hidden_size, hidden_size, init_method
      )
      self.layernorm = get_layer_norm(
          hidden_size, eps=layernorm_epsilon
      )
      self.gelu = torch.nn.functional.gelu
      if use_openai_gelu:
          self.gelu = openai_gelu
      elif onnx_safe:
          self.gelu = erf_gelu
  
file_path: NeMo/nemo/collections/nlp/models/language_modeling/megatron/bert_model.py
imports: import torch
  from nemo.collections.nlp.modules.common.megatron.language_model import get_language_model
  from nemo.collections.nlp.modules.common.megatron.module import MegatronModule
  from nemo.collections.nlp.modules.common.megatron.transformer import get_layer_norm
  from nemo.collections.nlp.modules.common.megatron.utils import ApexGuardDefaults, build_position_ids, erf_gelu, get_linear_layer, init_method_normal, openai_gelu, parallel_lm_logits, scaled_init_method_normal
label_desc: Use image node_id 1 to create a new BertLMHead object from inherited base classes: MegatronModule with example: obj = BertLMHead(config, mpu_vocab_size, hidden_size, init_method, layernorm_epsilon, parallel_output, use_openai_gelu, onnx_safe)