title: How to organize multiple functions returning same token with PLY?
body: <p>Let's say I would like to have just one PLY token - <code>'INTEGER'</code>. However I would like to be able to parse typical C-style literals in different bases, so effectively I would like to be able to parse strings like <code>0b10</code> (or <code>0B10</code>), <code>010</code>, <code>10</code> and <code>0x10</code> (or <code>0X10</code>). As I don't really care what was the "input format", I would like to just have the value as <code>int</code> in Python.</p>
  
  <p>However handling all these 4 cases in single function is not especially convenient... First or all the regex becomes pretty long: <code>r'0[0-7]+|0[bB][01]+|0[xX][0-9a-fA-F]+|[0-9]'</code>. But this is the smaller issue - the code of the function has to deal with multitudes of combinations to know which base to use, as the string starting with <code>0</code> in reality can be only single character, so checking further cases (next character is <code>x</code>, <code>X</code>, <code>b</code> or <code>B</code>) has to also take the length into account.</p>
  
  <p>So I would just prefer to have that as 4 separate functions, but all returning the same <code>'INTEGER'</code> type of token. I would prefer to not introduce <code>BINARY_INTEGER</code>, <code>OCTAL_INTEGER</code>, <code>DECIMAL_INTEGER</code> and <code>HEXADECIMAL_INTEGER</code>, because this would needlessly complicate the parser (or maybe I'm overthinking that?).</p>
  
  <p>I was wondering whether there's something smarter to do than just forcing <code>token.type</code> to be <code>'INTEGER'</code> in four "free" functions? Something other than:</p>
  
  <pre><code>def t_BINARY_LITERAL(t):
      r'0[bB][01]+'
      t.value = int(t.value[2:], 2)
      t.type = 'INTEGER'
      return t
  
  def t_OCTAL_LITERAL(t):
      r'0[0-7]+'
      t.value = int(t.value[1:], 8)
      t.type = 'INTEGER'
      return t
  
  def t_DECIMAL_LITERAL(t):
      r'[0-9]+'
      t.value = int(t.value, 10)
      t.type = 'INTEGER'
      return t
  
  def t_HEXADECIMAL_LITERAL(t):
      r'0[xX][0-9a-fA-F]+'
      t.value = int(t.value[2:], 16)
      t.type = 'INTEGER'
      return t
  </code></pre>