class_docstr: Class that implements a lexer for a given environment. Automatically
  created by the environment class, usually you don't have to do that.
  
  Note that the lexer is not automatically bound to an environment.
  Multiple environments can share the same lexer.
class_docstr_tok: ["Class","that","implements","a","lexer","for","a","given","environment",".","Automatically","created","by","the","environment","class",",","usually","you","do","n't","have","to","do","that",".","Note","that","the","lexer","is","not","automatically","bound","to","an","environment",".","Multiple","environments","can","share","the","same","lexer","."]
docstr: Calls tokeniter + tokenize and wraps it in a token stream.
docstr_tok: ["Calls","tokeniter","+","tokenize","and","wraps","it","in","a","token","stream","."]
code: def tokenize(self, source, name=None, filename=None, state=None):
      """Calls tokeniter + tokenize and wraps it in a token stream."""
      stream = self.tokeniter(source, name, filename, state)
      return TokenStream(
          self.wrap(stream, name, filename), name, filename
      )
  
imports: import re
  from ast import literal_eval
  from collections import deque
  from operator import itemgetter
  from ._compat import implements_iterator
  from ._compat import intern
  from ._compat import iteritems
  from ._compat import text_type
  from .exceptions import TemplateSyntaxError
  from .utils import LRUCache
label_desc: Use image node_id 3 for calling the Lexer obj's underlying member method code with example usage: obj.tokenize(source, name, filename, state) and returns: TokenStream