class_docstr: For `Modula-2 <http://www.modula2.org/>`_ source code.
  
  The Modula-2 lexer supports several dialects.  By default, it operates in
  fallback mode, recognising the *combined* literals, punctuation symbols
  and operators of all supported dialects, and the *combined* reserved words
  and builtins of PIM Modula-2, ISO Modula-2 and Modula-2 R10, while not
  differentiating between library defined identifiers.
  
  To select a specific dialect, a dialect option may be passed
  or a dialect tag may be embedded into a source file.
  
  Dialect Options:
  
  `m2pim`
      Select PIM Modula-2 dialect.
  `m2iso`
      Select ISO Modula-2 dialect.
  `m2r10`
      Select Modula-2 R10 dialect.
  `objm2`
      Select Objective Modula-2 dialect.
  
  The PIM and ISO dialect options may be qualified with a language extension.
  
  Language Extensions:
  
  `+aglet`
      Select Aglet Modula-2 extensions, available with m2iso.
  `+gm2`
      Select GNU Modula-2 extensions, available with m2pim.
  `+p1`
      Select p1 Modula-2 extensions, available with m2iso.
  `+xds`
      Select XDS Modula-2 extensions, available with m2iso.
  
  
  Passing a Dialect Option via Unix Commandline Interface
  
  Dialect options may be passed to the lexer using the `dialect` key.
  Only one such option should be passed. If multiple dialect options are
  passed, the first valid option is used, any subsequent options are ignored.
  
  Examples:
  
  `$ pygmentize -O full,dialect=m2iso -f html -o /path/to/output /path/to/input`
      Use ISO dialect to render input to HTML output
  `$ pygmentize -O full,dialect=m2iso+p1 -f rtf -o /path/to/output /path/to/input`
      Use ISO dialect with p1 extensions to render input to RTF output
  
  
  Embedding a Dialect Option within a source file
  
  A dialect option may be embedded in a source file in form of a dialect
  tag, a specially formatted comment that specifies a dialect option.
  
  Dialect Tag EBNF::
  
     dialectTag :
         OpeningCommentDelim Prefix dialectOption ClosingCommentDelim ;
  
     dialectOption :
         'm2pim' | 'm2iso' | 'm2r10' | 'objm2' |
         'm2iso+aglet' | 'm2pim+gm2' | 'm2iso+p1' | 'm2iso+xds' ;
  
     Prefix : '!' ;
  
     OpeningCommentDelim : '(*' ;
  
     ClosingCommentDelim : '*)' ;
  
  No whitespace is permitted between the tokens of a dialect tag.
  
  In the event that a source file contains multiple dialect tags, the first
  tag that contains a valid dialect option will be used and any subsequent
  dialect tags will be ignored.  Ideally, a dialect tag should be placed
  at the beginning of a source file.
  
  An embedded dialect tag overrides a dialect option set via command line.
  
  Examples:
  
  ``(*!m2r10*) DEFINITION MODULE Foobar; ...``
      Use Modula2 R10 dialect to render this source file.
  ``(*!m2pim+gm2*) DEFINITION MODULE Bazbam; ...``
      Use PIM dialect with GNU extensions to render this source file.
  
  
  Algol Publication Mode:
  
  In Algol publication mode, source text is rendered for publication of
  algorithms in scientific papers and academic texts, following the format
  of the Revised Algol-60 Language Report.  It is activated by passing
  one of two corresponding styles as an option:
  
  `algol`
      render reserved words lowercase underline boldface
      and builtins lowercase boldface italic
  `algol_nu`
      render reserved words lowercase boldface (no underlining)
      and builtins lowercase boldface italic
  
  The lexer automatically performs the required lowercase conversion when
  this mode is activated.
  
  Example:
  
  ``$ pygmentize -O full,style=algol -f latex -o /path/to/output /path/to/input``
      Render input file in Algol publication mode to LaTeX output.
  
  
  Rendering Mode of First Class ADT Identifiers:
  
  The rendering of standard library first class ADT identifiers is controlled
  by option flag "treat_stdlib_adts_as_builtins".
  
  When this option is turned on, standard library ADT identifiers are rendered
  as builtins.  When it is turned off, they are rendered as ordinary library
  identifiers.
  
  `treat_stdlib_adts_as_builtins` (default: On)
  
  The option is useful for dialects that support ADTs as first class objects
  and provide ADTs in the standard library that would otherwise be built-in.
  
  At present, only Modula-2 R10 supports library ADTs as first class objects
  and therefore, no ADT identifiers are defined for any other dialects.
  
  Example:
  
  ``$ pygmentize -O full,dialect=m2r10,treat_stdlib_adts_as_builtins=Off ...``
      Render standard library ADTs as ordinary library types.
  
  .. versionadded:: 1.3
  
  .. versionchanged:: 2.1
     Added multi-dialect support.
code: def get_tokens_unprocessed(self, text):
      for index, token, value in RegexLexer.get_tokens_unprocessed(
          self, text
      ):
          #
          # check for dialect tag if dialect has not been set by tag
          if not self.dialect_set_by_tag and token == Comment.Special:
              indicated_dialect = self.get_dialect_from_dialect_tag(
                  value
              )
              if indicated_dialect != "unknown":
                  # token is a dialect indicator
                  # reset reserved words and builtins
                  self.set_dialect(indicated_dialect)
                  self.dialect_set_by_tag = True
          #
          # check for reserved words, predefined and stdlib identifiers
          if token is Name:
              if value in self.reserved_words:
                  token = Keyword.Reserved
                  if self.algol_publication_mode:
                      value = value.lower()
              #
              elif value in self.builtins:
                  token = Name.Builtin
                  if self.algol_publication_mode:
                      value = value.lower()
              #
              elif value in self.pseudo_builtins:
                  token = Name.Builtin.Pseudo
                  if self.algol_publication_mode:
                      value = value.lower()
              #
              elif value in self.adts:
                  if not self.treat_stdlib_adts_as_builtins:
                      token = Name.Namespace
                  else:
                      token = Name.Builtin.Pseudo
                      if self.algol_publication_mode:
                          value = value.lower()
              #
              elif value in self.modules:
                  token = Name.Namespace
              #
              elif value in self.types:
                  token = Name.Class
              #
              elif value in self.procedures:
                  token = Name.Function
              #
              elif value in self.variables:
                  token = Name.Variable
              #
              elif value in self.constants:
                  token = Name.Constant
          #
          elif token in Number:
              #
              # mark prefix number literals as error for PIM and ISO dialects
              if self.dialect not in ("unknown", "m2r10", "objm2"):
                  if "'" in value or value[0:2] in ("0b", "0x", "0u"):
                      token = Error
              #
              elif self.dialect in ("m2r10", "objm2"):
                  # mark base-8 number literals as errors for M2 R10 and ObjM2
                  if token is Number.Oct:
                      token = Error
                  # mark suffix base-16 literals as errors for M2 R10 and ObjM2
                  elif token is Number.Hex and "H" in value:
                      token = Error
                  # mark real numbers with E as errors for M2 R10 and ObjM2
                  elif token is Number.Float and "E" in value:
                      token = Error
          #
          elif token in Comment:
              #
              # mark single line comment as error for PIM and ISO dialects
              if token is Comment.Single:
                  if self.dialect not in ("unknown", "m2r10", "objm2"):
                      token = Error
              #
              if token is Comment.Preproc:
                  # mark ISO pragma as error for PIM dialects
                  if value.startswith("<*") and self.dialect.startswith(
                      "m2pim"
                  ):
                      token = Error
                  # mark PIM pragma as comment for other dialects
                  elif (
                      value.startswith("(*$")
                      and self.dialect != "unknown"
                      and not self.dialect.startswith("m2pim")
                  ):
                      token = Comment.Multiline
          #
          else:  # token is neither Name nor Comment
              #
              # mark lexemes matching the dialect's error token set as errors
              if value in self.lexemes_to_reject:
                  token = Error
              #
              # substitute lexemes when in Algol mode
              if self.algol_publication_mode:
                  if value == "#":
                      value = "≠"
                  elif value == "<=":
                      value = "≤"
                  elif value == ">=":
                      value = "≥"
                  elif value == "==":
                      value = "≡"
                  elif value == "*.":
                      value = "•"
  
          # return result
          yield index, token, value
  
file_path: wandb/wandb/vendor/pygments/lexers/modula2.py
imports: import re
  from pygments.lexer import RegexLexer, include
  from pygments.util import get_bool_opt, get_list_opt
  from pygments.token import Text, Comment, Operator, Keyword, Name, String, Number, Punctuation, Error
label_desc: Use image node_id 4 for calling the Modula2Lexer obj's underlying member method code with example usage: obj.get_tokens_unprocessed(text) without return types