class_docstr: Recreate a token stream formatted with the `RawTokenFormatter`.  This
  lexer raises exceptions during parsing if the token stream in the
  file is malformed.
  
  Additional options accepted:
  
  `compress`
      If set to ``"gz"`` or ``"bz2"``, decompress the token stream with
      the given compression algorithm before lexing (default: ``""``).
class_docstr_tok: ["Recreate","a","token","stream","formatted","with","the","`","RawTokenFormatter","`",".","This","lexer","raises","exceptions","during","parsing","if","the","token","stream","in","the","file","is","malformed",".","Additional","options","accepted",":","`","compress","`","If","set","to","``","``","gz","''","``","or","``","``","bz2","''","``",",","decompress","the","token","stream","with","the","given","compression","algorithm","before","lexing","(","default",":","``","``","''","``",")","."]
code: def get_tokens(self, text):
      if isinstance(text, text_type):
          # raw token stream never has any non-ASCII characters
          text = text.encode("ascii")
      if self.compress == "gz":
          import gzip
  
          gzipfile = gzip.GzipFile("", "rb", 9, BytesIO(text))
          text = gzipfile.read()
      elif self.compress == "bz2":
          import bz2
  
          text = bz2.decompress(text)
  
      # do not call Lexer.get_tokens() because we do not want Unicode
      # decoding to occur, and stripping is not optional.
      text = text.strip(b"\n") + b"\n"
      for i, t, v in self.get_tokens_unprocessed(text):
          yield t, v
  
file_path: wandb/wandb/vendor/pygments/lexers/special.py
imports: import re
  from pygments.lexer import Lexer
  from pygments.token import Token, Error, Text
  from pygments.util import get_choice_opt, text_type, BytesIO
label_desc: Use image node_id 2 for calling the RawTokenLexer obj's underlying member method code with example usage: obj.get_tokens(text) without return types