class_docstr: String with various indexes.
docstr: Initializer.
  
  Args:
      raw_string: string with raw text in it
      split_expression: Regex string or callable. If regex string, will be used with re.split.
          If callable, the function should return a list of tokens.
      bow: if True, a word is the same everywhere in the text - i.e. we
           will index multiple occurrences of the same word. If False,
           order matters, so that the same word will have different ids
           according to position.
      mask_string: If not None, replace words with this if bow=False
          if None, default value is UNKWORDZ
code: def __init__(
      self,
      raw_string,
      split_expression=r"\W+",
      bow=True,
      mask_string=None,
  ):
      """Initializer.
  
      Args:
          raw_string: string with raw text in it
          split_expression: Regex string or callable. If regex string, will be used with re.split.
              If callable, the function should return a list of tokens.
          bow: if True, a word is the same everywhere in the text - i.e. we
               will index multiple occurrences of the same word. If False,
               order matters, so that the same word will have different ids
               according to position.
          mask_string: If not None, replace words with this if bow=False
              if None, default value is UNKWORDZ
      """
      self.raw = raw_string
      self.mask_string = (
          "UNKWORDZ" if mask_string is None else mask_string
      )
  
      if callable(split_expression):
          tokens = split_expression(self.raw)
          self.as_list = self._segment_with_tokens(self.raw, tokens)
          tokens = set(tokens)
  
          def non_word(string):
              return string not in tokens
  
      else:
          # with the split_expression as a non-capturing group (?:), we don't need to filter out
          # the separator character from the split results.
          splitter = re.compile(r"(%s)|$" % split_expression)
          self.as_list = [s for s in splitter.split(self.raw) if s]
          non_word = splitter.match
  
      self.as_np = np.array(self.as_list)
      self.string_start = np.hstack(
          ([0], np.cumsum([len(x) for x in self.as_np[:-1]]))
      )
      vocab = {}
      self.inverse_vocab = []
      self.positions = []
      self.bow = bow
      non_vocab = set()
      for i, word in enumerate(self.as_np):
          if word in non_vocab:
              continue
          if non_word(word):
              non_vocab.add(word)
              continue
          if bow:
              if word not in vocab:
                  vocab[word] = len(vocab)
                  self.inverse_vocab.append(word)
                  self.positions.append([])
              idx_word = vocab[word]
              self.positions[idx_word].append(i)
          else:
              self.inverse_vocab.append(word)
              self.positions.append(i)
      if not bow:
          self.positions = np.array(self.positions)
  
imports: from functools import partial
  import itertools
  import json
  import re
  import numpy
  import scipy
  import sklearn
  from sklearn.utils import check_random_state
  from .None import explanation
  from .None import lime_base
label_desc: Use image node_id 1 to create a new IndexedString object from inherited base classes: object with example: obj = IndexedString(raw_string, split_expression, bow, mask_string)