class_docstr: Class that implements a lexer for a given environment. Automatically
  created by the environment class, usually you don't have to do that.
  
  Note that the lexer is not automatically bound to an environment.
  Multiple environments can share the same lexer.
class_docstr_tok: ["Class","that","implements","a","lexer","for","a","given","environment",".","Automatically","created","by","the","environment","class",",","usually","you","do","n't","have","to","do","that",".","Note","that","the","lexer","is","not","automatically","bound","to","an","environment",".","Multiple","environments","can","share","the","same","lexer","."]
docstr: This is called with the stream as returned by `tokenize` and wraps
  every token in a :class:`Token` and converts the value.
docstr_tok: ["This","is","called","with","the","stream","as","returned","by","`","tokenize","`","and","wraps","every","token","in","a",":","class",":","`","Token","`","and","converts","the","value","."]
code: def wrap(self, stream, name=None, filename=None):
      """This is called with the stream as returned by `tokenize` and wraps
      every token in a :class:`Token` and converts the value.
      """
      for lineno, token, value in stream:
          if token in ignored_tokens:
              continue
          elif token == TOKEN_LINESTATEMENT_BEGIN:
              token = TOKEN_BLOCK_BEGIN
          elif token == TOKEN_LINESTATEMENT_END:
              token = TOKEN_BLOCK_END
          # we are not interested in those tokens in the parser
          elif token in (TOKEN_RAW_BEGIN, TOKEN_RAW_END):
              continue
          elif token == TOKEN_DATA:
              value = self._normalize_newlines(value)
          elif token == "keyword":
              token = value
          elif token == TOKEN_NAME:
              value = str(value)
              if check_ident and not value.isidentifier():
                  raise TemplateSyntaxError(
                      "Invalid character in identifier",
                      lineno,
                      name,
                      filename,
                  )
          elif token == TOKEN_STRING:
              # try to unescape string
              try:
                  value = (
                      self._normalize_newlines(value[1:-1])
                      .encode("ascii", "backslashreplace")
                      .decode("unicode-escape")
                  )
              except Exception as e:
                  msg = str(e).split(":")[-1].strip()
                  raise TemplateSyntaxError(msg, lineno, name, filename)
          elif token == TOKEN_INTEGER:
              value = int(value.replace("_", ""))
          elif token == TOKEN_FLOAT:
              # remove all "_" first to support more Python versions
              value = literal_eval(value.replace("_", ""))
          elif token == TOKEN_OPERATOR:
              token = operators[value]
          yield Token(lineno, token, value)
  
imports: import re
  from ast import literal_eval
  from collections import deque
  from operator import itemgetter
  from ._compat import implements_iterator
  from ._compat import intern
  from ._compat import iteritems
  from ._compat import text_type
  from .exceptions import TemplateSyntaxError
  from .utils import LRUCache
label_desc: Use image node_id 4 for calling the Lexer obj's underlying member method code with example usage: obj.wrap(stream, name, filename) without return types