class_docstr: This lexer takes one lexer as argument, the lexer for the language
  being formatted, and the left and right delimiters for escaped text.
  
  First everything is scanned using the language lexer to obtain
  strings and comments. All other consecutive tokens are merged and
  the resulting text is scanned for escaped segments, which are given
  the Token.Escape type. Finally text that is not escaped is scanned
  again with the language lexer.
class_docstr_tok: ["This","lexer","takes","one","lexer","as","argument",",","the","lexer","for","the","language","being","formatted",",","and","the","left","and","right","delimiters","for","escaped","text",".","First","everything","is","scanned","using","the","language","lexer","to","obtain","strings","and","comments",".","All","other","consecutive","tokens","are","merged","and","the","resulting","text","is","scanned","for","escaped","segments",",","which","are","given","the","Token.Escape","type",".","Finally","text","that","is","not","escaped","is","scanned","again","with","the","language","lexer","."]
code: def get_tokens_unprocessed(self, text):
      buf = ""
      idx = 0
      for i, t, v in self.lang.get_tokens_unprocessed(text):
          if t in Token.Comment or t in Token.String:
              if buf:
                  for x in self.get_tokens_aux(idx, buf):
                      yield x
                  buf = ""
              yield i, t, v
          else:
              if not buf:
                  idx = i
              buf += v
      if buf:
          for x in self.get_tokens_aux(idx, buf):
              yield x
  
file_path: catboost/contrib/python/Pygments/py2/pygments/formatters/latex.py
imports: from __future__ import division
  from pygments.formatter import Formatter
  from pygments.lexer import Lexer
  from pygments.token import Token, STANDARD_TYPES
  from pygments.util import get_bool_opt, get_int_opt, StringIO, xrange, iteritems
label_desc: Use image node_id 2 for calling the LatexEmbeddedLexer obj's underlying member method code with example usage: obj.get_tokens_unprocessed(text) without return types