class_docstr: Dataset that returns tokenized prompts from a stream of text files.
      Args:
          prompt_tokenizer (PromptTokenizingStrategy): The prompt tokenizing method for processing the data.
          dataset (dataset.Dataset): Dataset with text files.
code: def process(self, dataset):
      features = dataset.features.keys()
      num_proc = (
          min(64, self.process_count)
          if self.process_count
          else min(64, os.cpu_count())
      )
      map_kwargs = {}
      if self.prompt_tokenizer.supports_batched:
          map_kwargs["batched"] = True
          map_kwargs["batch_size"] = 100
      return dataset.map(
          self.prompt_tokenizer.tokenize_prompt,
          num_proc=num_proc,
          remove_columns=features,
          **map_kwargs,
      )
  
file_path: axolotl/src/axolotl/datasets.py
imports: import logging
  import os
  from typing import List, Optional
  import torch
  from datasets import Dataset, IterableDataset
  from .prompt_tokenizers import PromptTokenizingStrategy
label_desc: Use image node_id 2 for calling the TokenizedPromptDataset obj's underlying member method code with example usage: obj.process(dataset) and returns: dataset