class_docstr: A Scanner is used to read tokens from a stream of characters
  using the token set specified by a Plex.Lexicon.
  
  Constructor:
  
    Scanner(lexicon, stream, name = '')
  
      See the docstring of the __init__ method for details.
  
  Methods:
  
    See the docstrings of the individual methods for more
    information.
  
    read() --> (value, text)
      Reads the next lexical token from the stream.
  
    position() --> (name, line, col)
      Returns the position of the last token read using the
      read() method.
  
    begin(state_name)
      Causes scanner to change state.
  
    produce(value [, text])
      Causes return of a token value to the caller of the
      Scanner.
docstr: Read the next input sequence recognised by the machine
  and return (text, action). Returns ('', None) on end of
  file.
docstr_tok: ["Read","the","next","input","sequence","recognised","by","the","machine","and","return","(","text",",","action",")",".","Returns","(","``",",","None",")","on","end","of","file","."]
code: def scan_a_token(self):
      """
      Read the next input sequence recognised by the machine
      and return (text, action). Returns ('', None) on end of
      file.
      """
      self.start_pos = self.cur_pos
      self.start_line = self.cur_line
      self.start_col = self.cur_pos - self.cur_line_start
      action = self.run_machine_inlined()
      if action is not None:
          if self.trace:
              print(
                  "Scanner: read: Performing %s %d:%d"
                  % (action, self.start_pos, self.cur_pos)
              )
          text = self.buffer[
              self.start_pos
              - self.buf_start_pos : self.cur_pos
              - self.buf_start_pos
          ]
          return (text, action)
      else:
          if self.cur_pos == self.start_pos:
              if self.cur_char is EOL:
                  self.next_char()
              if self.cur_char is None or self.cur_char is EOF:
                  return ("", None)
          raise Errors.UnrecognizedInput(self, self.state_name)
  
file_path: catboost/contrib/tools/cython/Cython/Plex/Scanners.py
imports: from __future__ import absolute_import
  import cython
  from .None import Errors
  from .Regexps import BOL, EOL, EOF
label_desc: Use image node_id 3 for calling the Scanner obj's underlying member method code with example usage: obj.scan_a_token() and returns: text,  action, str,  None