class_docstr: A basic model agnostic tokenizer.
      
docstr: Tokenize the passed string, optionally returning the offsets of each token in the original string.
          
code: def __call__(self, s, return_offsets_mapping=True):
      """Tokenize the passed string, optionally returning the offsets of each token in the original string."""
      pos = 0
      offset_ranges = []
      input_ids = []
      for m in re.finditer(self.split_pattern, s):
          start, end = m.span(0)
          offset_ranges.append((pos, start))
          input_ids.append(s[pos:start])
          pos = end
      if pos != len(s):
          offset_ranges.append((pos, len(s)))
          input_ids.append(s[pos:])
  
      out = {}
      out["input_ids"] = input_ids
      if return_offsets_mapping:
          out["offset_mapping"] = offset_ranges
      return out
  
imports: import math
  import re
  import numpy
  from .._serializable import Deserializer, Serializer
  from ..utils import safe_isinstance
  from ..utils.transformers import SENTENCEPIECE_TOKENIZERS, getattr_silent, parse_prefix_suffix_for_tokenizer
  from ._masker import Masker
label_desc: Use image node_id 2 for calling the SimpleTokenizer obj's underlying member method code with example usage: obj.__call__(s, return_offsets_mapping) and returns: out