class_docstr: Language model embeddings.
  
  Arguments:
      hidden_size: hidden size
      vocab_size: vocabulary size
      max_sequence_length: maximum size of sequence. This
                           is used for positional embedding
      embedding_dropout_prob: dropout probability for embeddings
      init_method: weight initialization method
      num_tokentypes: size of the token-type embeddings. 0 value
                      will ignore this embedding
class_docstr_tok: ["Language","model","embeddings",".","Arguments",":","hidden_size",":","hidden","size","vocab_size",":","vocabulary","size","max_sequence_length",":","maximum","size","of","sequence",".","This","is","used","for","positional","embedding","embedding_dropout_prob",":","dropout","probability","for","embeddings","init_method",":","weight","initialization","method","num_tokentypes",":","size","of","the","token-type","embeddings",".","0","value","will","ignore","this","embedding"]
docstr: Add token-type embedding. This function is provided so we can add
  token-type embeddings in case the pretrained model does not have it.
  This allows us to load the model normally and then add this embedding.
docstr_tok: ["Add","token-type","embedding",".","This","function","is","provided","so","we","can","add","token-type","embeddings","in","case","the","pretrained","model","does","not","have","it",".","This","allows","us","to","load","the","model","normally","and","then","add","this","embedding","."]
code: def add_tokentype_embeddings(self, num_tokentypes):
      """Add token-type embedding. This function is provided so we can add
      token-type embeddings in case the pretrained model does not have it.
      This allows us to load the model normally and then add this embedding.
      """
      if self.tokentype_embeddings is not None:
          raise Exception("tokentype embeddings is already initialized")
      if torch.distributed.get_rank() == 0:
          print(
              "adding embedding for {} tokentypes".format(
                  num_tokentypes
              ),
              flush=True,
          )
      self.num_tokentypes = num_tokentypes
      self.tokentype_embeddings = torch.nn.Embedding(
          num_tokentypes, self.hidden_size
      )
  
imports: import torch
  import torch.nn.init
  from torch import Tensor
  from torch import nn
  from torch.nn import functional
  from torch.nn.parameter import Parameter
  from colossalai.legacy.context import ParallelMode, seed
  from colossalai.legacy.core import global_context
  from colossalai.legacy.nn.layer.base_layer import ParallelLayer
  from colossalai.legacy.nn.layer.parallel_1d._utils import gather_forward_split_backward, reduce_grad, reduce_input
  from colossalai.legacy.nn.layer.parallel_1d.layers import Linear1D_Row
  from colossalai.legacy.nn.layer.utils import divide
  from colossalai.legacy.registry import LAYERS, LOSSES
  from colossalai.utils import get_current_device
label_desc: Use image node_id 3 for calling the VocabParallelEmbedding obj's underlying member method code with example usage: obj.add_tokentype_embeddings(num_tokentypes) without return types