title: how to distinguish tokens which have similar patterns in Lexer, but they occur in different contexts in the parser
body: <p>I have two pretty similar patterns in Lexer.x first for numbers second for byte. Here they are.</p>
  <pre><code>$digit=0-9
  $byte=[a-f0-9]
  
  
      $digit+                       { \s -&gt; TNum  (readRational s) }
      $digit+.$digit+               { \s -&gt; TNum  (readRational s) }
      $digit+.$digit+e$digit+       { \s -&gt; TNum  (readRational s) }
      $digit+e$digit+               { \s -&gt; TNum  (readRational s) }
      $byte$byte                        { \s -&gt; TByte (encodeUtf8(pack s))     }
  </code></pre>
  <p>I have Parser.y</p>
  <pre><code>%token
  
          cnst                            { TNum  $$}
          byte                            { TByte  $$}
          '['                            { TOSB     }    
          ']'                            { TCSB     }
  
  %%
  
  Expr: 
   '[' byte ']' {$1}
  | const {$1}
  </code></pre>
  <p>when I write, I got.</p>
  <pre><code>[ 11 ] parse error
  11 ok
  </code></pre>
  <p>but when I put byte pattern in Lexer before numbers</p>
  <pre><code>$digit=0-9
  $byte=[a-f0-9]
  
      $byte$byte                        { \s -&gt; TByte (encodeUtf8(pack s))     }
      $digit+                       { \s -&gt; TNum  (readRational s) }
      $digit+.$digit+               { \s -&gt; TNum  (readRational s) }
      $digit+.$digit+e$digit+       { \s -&gt; TNum  (readRational s) }
      $digit+e$digit+               { \s -&gt; TNum  (readRational s) }
  
  </code></pre>
  <p>I got</p>
  <pre><code>[ 11 ] ok
  11 parse error
  </code></pre>
  <p>I think that happens because Lexer makes tokens from string and then gives them to parser.
  And when parser wait for byte token it got number token and parser don't have opportunity to make from this value another token.
  What I should do in this situation?</p>