class_docstr: Recreate a token stream formatted with the `RawTokenFormatter`.
  
  Additional options accepted:
  
  `compress`
      If set to ``"gz"`` or ``"bz2"``, decompress the token stream with
      the given compression algorithm before lexing (default: ``""``).
code: def get_tokens_unprocessed(self, text):
      length = 0
      for match in line_re.finditer(text):
          try:
              ttypestr, val = match.group().rstrip().split("\t", 1)
              ttype = _ttype_cache.get(ttypestr)
              if not ttype:
                  ttype = Token
                  ttypes = ttypestr.split(".")[1:]
                  for ttype_ in ttypes:
                      if not ttype_ or not ttype_[0].isupper():
                          raise ValueError("malformed token name")
                      ttype = getattr(ttype, ttype_)
                  _ttype_cache[ttypestr] = ttype
              val = ast.literal_eval(val)
              if not isinstance(val, str):
                  raise ValueError("expected str")
          except (SyntaxError, ValueError):
              val = match.group()
              ttype = Error
          yield length, ttype, val
          length += len(val)
  
file_path: catboost/contrib/python/Pygments/py3/pygments/lexers/special.py
imports: import ast
  from pygments.lexer import Lexer, line_re
  from pygments.token import Token, Error, Text, Generic
  from pygments.util import get_choice_opt
label_desc: Use image node_id 3 for calling the RawTokenLexer obj's underlying member method code with example usage: obj.get_tokens_unprocessed(text) without return types