title: JFlex match nested comments as one token
body: <p>In Mathematica a comment starts with <code>(*</code> and ends with <code>*)</code> and comments can be nested. My current approach of scanning a comment with JFlex contains the following code</p>
  
  <pre><code>%xstate IN_COMMENT
  
  "(*"  { yypushstate(IN_COMMENT); return MathematicaElementTypes.COMMENT;}
  
  &lt;IN_COMMENT&gt; {
    "(*"        {yypushstate(IN_COMMENT); return MathematicaElementTypes.COMMENT;}
    [^\*\)\(]*  {return MathematicaElementTypes.COMMENT;}
    "*)"        {yypopstate(); return MathematicaElementTypes.COMMENT;}
    [\*\)\(]    {return MathematicaElementTypes.COMMENT;}
    .           {return MathematicaElementTypes.BAD_CHARACTER;}
  }
  </code></pre>
  
  <p>where the methods <code>yypushstate</code> and <code>yypopstate</code> are defined as</p>
  
  <pre><code>private final LinkedList&lt;Integer&gt; states = new LinkedList();
  
  private void yypushstate(int state) {
      states.addFirst(yystate());
      yybegin(state);
  }
  private void yypopstate() {
      final int state = states.removeFirst();
      yybegin(state);
  }
  </code></pre>
  
  <p>to give me the opportunity to track how many nested levels of comment I'm dealing with.</p>
  
  <p>Unfortunately, this results in several <code>COMMENT</code> tokens for one comment, because I have to match nested comment starts and comment ends. </p>
  
  <p><strong>Question:</strong> Is it possible with JFlex to use its API with methods like <code>yypushback</code> or <code>advance()</code> etc. to return exactly one token over the whole comment range, even if comments are nested?</p>