title: Handle a token one of two ways depending on context
body: <p>Is it possible to handle the same lexical token differently depending on context? The docs says there are <a href="https://github.com/antlr/antlr4/blob/master/doc/lexer-rules.md#mode-pushmode-popmode-and-more" rel="nofollow noreferrer"><code>Lexical Modes</code></a> as a way to handle this, and was wondering if that approach could be used to handle this.</p>
  <p>Here is an example. Given the following SQL statement:</p>
  <pre><code>select 1.2, 1.2e, .2, 1.e, tbl, tbl.select
  </code></pre>
  <p>Notice that the <code>SELECT</code> token at the beginning is a reserved keyword, but when the reserved token is preceded by a <code>.</code> it is part of a &quot;path expression&quot; and so is not a reserved keyword. What I want to do is -- ignoring numbers -- I want the first <code>select</code> to be captured as the <code>select</code> token and the second <code>select</code> to be captured as an <code>identifier</code>. Is this possible? Here is an example in antlr:</p>
  <pre><code>grammar DBParser;
  options { caseInsensitive=true; }
  
  statement:select_statement EOF;
  
  select_statement
      : SELECT expr (COMMA expr)*
      ;
  
  expr
      : NUMBER
      | IDENTIFIER
      ;
  
  
  COMMA: ',';
  
  // 1. Pick up numbers first (simplified number format for question)
  NUMBER: [0-9]* '.' [0-9]* 'e'?;
  
  // 2. Reserved Keyword section
  SELECT: 'SELECT';
  
  // 3. And if it's not a reserved keyword, pick it up as an identifier
  IDENTIFIER: [A-Z_] [A-Z_0-9]*;
  
  
  WHITESPACE: [ \t\r\n] -&gt; skip;
  </code></pre>
  <p>And here is the parse tree, where everything is ok until the <code>select</code> after the dot. Is there a rule I can set so that I can properly parse the above at the lexing stage?</p>
  <hr />
  <p><strong>Update</strong>: A more complex example might be something like:</p>
  <pre><code>select ([{'select': 1}])[0].select;
  </code></pre>
  <p>Where dot access may be (and often is) after a parenthesized expression, array access, function call, etc.</p>
  <p><a href="https://i.stack.imgur.com/rbmil.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/rbmil.png" alt="enter image description here" /></a></p>