class_docstr: Dataset to convert a text dataset to Bert tokens.
code: def __init__(self, text_dataset, max_seq_length):
      super().__init__()
  
      args = get_args()
  
      # Dataset, tokenizer.
      self.text_dataset = text_dataset
      self.bert_tokenizer = get_tokenizer()
  
      # Params to store.
      self.max_seq_length = max_seq_length
      self.seed = args.seed
      self.masked_lm_prob = args.mask_prob
  
      # Vocab stuff.
      self.vocab_id_list = list(self.bert_tokenizer.inv_vocab.keys())
      self.vocab_id_to_token_dict = self.bert_tokenizer.inv_vocab
      self.cls_id = self.bert_tokenizer.cls
      self.sep_id = self.bert_tokenizer.sep
      self.mask_id = self.bert_tokenizer.mask
      self.pad_id = self.bert_tokenizer.pad
  
file_path: megatron-lm/tools/bert_embedding/dataset.py
imports: import numpy
  import torch
  from megatron import get_args, get_tokenizer
  from megatron.data.bert_dataset import build_training_sample
label_desc: Use image node_id 1 to create a new BertEmbeddingDataset object from inherited base classes: torch.utils.data with example: obj = BertEmbeddingDataset(text_dataset, max_seq_length)