class_docstr: Megatron Bert pretraining.
  Model returns [batch, seq, hidden] shape
class_docstr_tok: ["Megatron","Bert","pretraining",".","Model","returns","[","batch",",","seq",",","hidden","]","shape"]
docstr: Builds the megatron core gpt transformer config for the model.
  For attributes in the nemo model config that are the same
  as the megatron core TransformerConfig, we will use the value from the nemo model config.
  For attributes in TransformerConfig that are not in the nemo model config, we add custom logic.
code: def build_transformer_config(self) -> TransformerConfig:
      """Builds the megatron core gpt transformer config for the model.
      For attributes in the nemo model config that are the same
      as the megatron core TransformerConfig, we will use the value from the nemo model config.
      For attributes in TransformerConfig that are not in the nemo model config, we add custom logic.
      """
      activation = self.cfg.get("activation", "gelu")
      assert (
          activation == "gelu"
      ), "Only gelu activation is support for BERT at the moment."
      transformer_config = super().build_transformer_config()
      return transformer_config
  
imports: import itertools
  from typing import Any, Dict, List, Optional
  import torch
  import torch.nn.functional
  from omegaconf.dictconfig import DictConfig
  from pytorch_lightning.trainer.trainer import Trainer
  from nemo.collections.nlp.data.language_modeling.megatron import dataset_utils
  from nemo.collections.nlp.data.language_modeling.megatron.data_samplers import MegatronPretrainingRandomSampler, MegatronPretrainingSampler
  from nemo.collections.nlp.models.language_modeling.megatron.bert_model import BertModel
  from nemo.collections.nlp.models.language_modeling.megatron_base_model import MegatronBaseModel
  from nemo.collections.nlp.modules.common.megatron.build_model import build_model
  from nemo.collections.nlp.modules.common.megatron.module import Float16Module
  from nemo.collections.nlp.modules.common.megatron.utils import ApexGuardDefaults, average_losses_across_data_parallel_group, get_params_for_weight_decay_optimization
  from nemo.collections.nlp.parts.nlp_overrides import GradScaler
  from nemo.collections.nlp.parts.utils_funcs import get_last_rank
  from nemo.core.classes.common import PretrainedModelInfo
  from nemo.core.neural_types import ChannelType, MaskType, NeuralType
  from nemo.utils import AppState, logging
label_desc: Use image node_id 35 for calling the MegatronBertModel obj's underlying member method code with example usage: obj.build_transformer_config() and returns: transformer_config