docstr: If max_position_embeddings <= 0, there's no position embeddings
  If type_vocab_size <= 0, there's no token type embeddings
docstr_tok: ["If","max_position_embeddings","<","=","0",",","there","'s","no","position","embeddings","If","type_vocab_size","<","=","0",",","there","'s","no","token","type","embeddings"]
code: def __init__(
      self,
      embed_dim,
      vocab_size,
      max_position_embeddings,
      type_vocab_size,
      padding_idx=None,
      device=None,
      dtype=None,
  ):
      """
      If max_position_embeddings <= 0, there's no position embeddings
      If type_vocab_size <= 0, there's no token type embeddings
      """
      factory_kwargs = {"device": device, "dtype": dtype}
      super().__init__()
      self.word_embeddings = nn.Embedding(
          vocab_size,
          embed_dim,
          padding_idx=padding_idx,
          **factory_kwargs,
      )
      self.max_position_embeddings = max_position_embeddings
      self.type_vocab_size = type_vocab_size
      if self.max_position_embeddings > 0:
          self.position_embeddings = nn.Embedding(
              max_position_embeddings, embed_dim, **factory_kwargs
          )
      if self.type_vocab_size > 0:
          self.token_type_embeddings = nn.Embedding(
              type_vocab_size, embed_dim, **factory_kwargs
          )
  
imports: import torch
  import torch.nn
  from einops import rearrange
  from torch import Tensor
  from flash_attn.utils.distributed import all_reduce, reduce_scatter
label_desc: Use image node_id 1 to create a new BertEmbeddings object from inherited base classes: nn with example: obj = BertEmbeddings(embed_dim, vocab_size, max_position_embeddings, type_vocab_size, padding_idx, device, dtype)