class_docstr: Iterable over a text dataset.
docstr: Remove 'endoftext' string.
docstr_tok: ["Remove","'endoftext","'","string","."]
code: def __iter__(self):
      """Remove 'endoftext' string."""
      for sample_idx in range(len(self.text_dataset)):
          sample = self.text_dataset[sample_idx]
          text = sample["text"].replace("<|endoftext|>", "")
          yield text
  
file_path: megatron-lm/tools/bert_embedding/huggingface.py
imports: import numpy
  import torch
  from tqdm import tqdm
  from .external_libs import transformers
label_desc: Use image node_id 2 for calling the IterableTextDataset obj's underlying member method code with example usage: obj.__iter__() without return types