class_docstr: Embedding parallelized in the vocabulary dimension.
  
  This is mainly adapted from torch.nn.Embedding and all the default
  values are kept.
  Arguments:
      num_embeddings: vocabulary size.
      embedding_dim: size of hidden state.
      init_method: method to initialize weights.
class_docstr_tok: ["Embedding","parallelized","in","the","vocabulary","dimension",".","This","is","mainly","adapted","from","torch.nn.Embedding","and","all","the","default","values","are","kept",".","Arguments",":","num_embeddings",":","vocabulary","size",".","embedding_dim",":","size","of","hidden","state",".","init_method",":","method","to","initialize","weights","."]
code: def __init__(
      self,
      num_embeddings,
      embedding_dim,
      dtype=torch.float,
      padding_idx: int = None,
      init_method=None,
  ):
      super(HiddenParallelEmbedding1D, self).__init__()
      # Keep the input dimensions.
      self.num_embeddings = num_embeddings
      self.embedding_dim = embedding_dim
      embed_dim_per_partition = divide(
          embedding_dim, gpc.tensor_parallel_size
      )
      # Set the details for compatibility.
      self.padding_idx = padding_idx
      self.max_norm = None
      self.norm_type = 2.0
      self.scale_grad_by_freq = False
      self.sparse = False
      self._weight = None
  
      # Allocate weights and initialize.
      factory_kwargs = {"device": get_current_device(), "dtype": dtype}
      self.weight = Parameter(
          torch.empty(
              num_embeddings, embed_dim_per_partition, **factory_kwargs
          )
      )
      init.uniform_(self.weight, -1, 1)
  
imports: import torch
  import torch.nn.init
  from torch import Tensor
  from torch import nn
  from torch.nn import functional
  from torch.nn.parameter import Parameter
  from colossalai.legacy.context import ParallelMode, seed
  from colossalai.legacy.core import global_context
  from colossalai.legacy.nn.layer.base_layer import ParallelLayer
  from colossalai.legacy.nn.layer.parallel_1d._utils import gather_forward_split_backward, reduce_grad, reduce_input
  from colossalai.legacy.nn.layer.parallel_1d.layers import Linear1D_Row
  from colossalai.legacy.nn.layer.utils import divide
  from colossalai.legacy.registry import LAYERS, LOSSES
  from colossalai.utils import get_current_device
label_desc: Use image node_id 1 to create a new HiddenParallelEmbedding1D object from inherited base classes: torch.nn with example: obj = HiddenParallelEmbedding1D(num_embeddings, embedding_dim, dtype, padding_idx, init_method)