class_docstr: A Scanner is used to read tokens from a stream of characters
  using the token set specified by a Plex.Lexicon.
  
  Constructor:
  
    Scanner(lexicon, stream, name = '')
  
      See the docstring of the __init__ method for details.
  
  Methods:
  
    See the docstrings of the individual methods for more
    information.
  
    read() --> (value, text)
      Reads the next lexical token from the stream.
  
    position() --> (name, line, col)
      Returns the position of the last token read using the
      read() method.
  
    begin(state_name)
      Causes scanner to change state.
  
    produce(value [, text])
      Causes return of a token value to the caller of the
      Scanner.
docstr: Scanner(lexicon, stream, name = '')
  
    |lexicon| is a Plex.Lexicon instance specifying the lexical tokens
    to be recognised.
  
    |stream| can be a file object or anything which implements a
    compatible read() method.
  
    |name| is optional, and may be the name of the file being
    scanned or any other identifying string.
docstr_tok: ["Scanner","(","lexicon",",","stream",",","name","=","``",")","|lexicon|","is","a","Plex.Lexicon","instance","specifying","the","lexical","tokens","to","be","recognised",".","|stream|","can","be","a","file","object","or","anything","which","implements","a","compatible","read","(",")","method",".","|name|","is","optional",",","and","may","be","the","name","of","the","file","being","scanned","or","any","other","identifying","string","."]
code: def __init__(self, lexicon, stream, name="", initial_pos=None):
      """
      Scanner(lexicon, stream, name = '')
  
        |lexicon| is a Plex.Lexicon instance specifying the lexical tokens
        to be recognised.
  
        |stream| can be a file object or anything which implements a
        compatible read() method.
  
        |name| is optional, and may be the name of the file being
        scanned or any other identifying string.
      """
      self.trace = 0
  
      self.buffer = ""
      self.buf_start_pos = 0
      self.next_pos = 0
      self.cur_pos = 0
      self.cur_line = 1
      self.start_pos = 0
      self.start_line = 0
      self.start_col = 0
      self.text = None
      self.state_name = None
  
      self.lexicon = lexicon
      self.stream = stream
      self.name = name
      self.queue = []
      self.initial_state = None
      self.begin("")
      self.next_pos = 0
      self.cur_pos = 0
      self.cur_line_start = 0
      self.cur_char = BOL
      self.input_state = 1
      if initial_pos is not None:
          self.cur_line, self.cur_line_start = (
              initial_pos[1],
              -initial_pos[2],
          )
  
file_path: catboost/contrib/tools/cython/Cython/Plex/Scanners.py
imports: from __future__ import absolute_import
  import cython
  from .None import Errors
  from .Regexps import BOL, EOL, EOF
label_desc: Use image node_id 1 to create a new Scanner object from inherited base classes: object with example: obj = Scanner(lexicon, stream, name, initial_pos)