class_docstr: A regular expression-based tokenizer at word boundary.
  This tokenizer default to support MegaMolBART.
  <https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/models/megamolbart>
code: def tokens_to_ids(self, token_data):
      if isinstance(token_data, str):
          token_data = [token_data]
  
      ids_list = []
      for tokens in token_data:
          ids = self.token_to_ids(tokens)
          ids_list.append(ids)
      return ids_list
  
imports: import os
  import re
  from typing import Optional
  import pandas
  from nemo.collections.common.tokenizers.char_tokenizer import TokenizerSpec
  from nemo.utils import logging
label_desc: Use image node_id 8 for calling the RegExTokenizer obj's underlying member method code with example usage: obj.tokens_to_ids(token_data) and returns: ids_list