class_docstr: Embedding parallelized in the vocabulary dimension.
  
  This is mainly adapted from torch.nn.Embedding and all the default
  values are kept.
  Arguments:
      num_embeddings: vocabulary size.
      embedding_dim: size of hidden state.
  
  Keyword Arguments:
      config: A megatron.core.ModelParallelConfig object
code: def forward(self, input_):
      assert not torch.any(
          (input_ < 0) | (input_ >= self.num_embeddings)
      ), "An input token is out of bounds of the embedding table"
      if self.tensor_model_parallel_size > 1:
          # Build the mask.
          input_mask = (input_ < self.vocab_start_index) | (
              input_ >= self.vocab_end_index
          )
          # Mask the input.
          masked_input = input_.clone() - self.vocab_start_index
          masked_input[input_mask] = 0
      else:
          masked_input = input_
      # Get the embeddings.
      output_parallel = self.weight[masked_input]
      # Mask the output embedding.
      if self.tensor_model_parallel_size > 1:
          output_parallel[input_mask, :] = 0.0
      # Reduce across all the model parallel GPUs.
      output = reduce_from_tensor_model_parallel_region(output_parallel)
      return output
  
imports: import math
  import os
  import warnings
  from typing import Callable, Optional
  import torch
  import torch.nn.functional
  import torch.nn.init
  from torch.cuda.amp import custom_bwd, custom_fwd
  from torch.nn.parameter import Parameter
  from megatron.core.model_parallel_config import ModelParallelConfig
  from megatron.core.parallel_state import get_global_memory_buffer, get_tensor_model_parallel_group, get_tensor_model_parallel_rank, get_tensor_model_parallel_world_size
  from .mappings import copy_to_tensor_model_parallel_region, gather_from_sequence_parallel_region, gather_from_tensor_model_parallel_region, reduce_from_tensor_model_parallel_region, reduce_scatter_to_sequence_parallel_region, scatter_to_tensor_model_parallel_region
  from .random import get_cuda_rng_tracker, get_expert_parallel_rng_tracker_name
  from .utils import VocabUtility, divide, split_tensor_along_last_dim
label_desc: Use image node_id 2 for calling the VocabParallelEmbedding obj's underlying member method code with example usage: obj.forward(input_) and returns: output