title: Programming a simple compiler
body: <p>I am writing a compiler for a simple language.</p>
  
  <p>I made a lexer/tokenizer that takes a file and prints the tokens in <code>stdout</code>.</p>
  
  <p>Now I want to to make the syntactical analysis, but I don't know how to modify my lexer in order to take the tokens as input.</p>
  
  <ul>
  <li>A linked list is extremely inefficient for large files (source files around 80MB take about 1.3GB of ram)</li>
  <li>I could modify my lexer to give the next token every time it is called (idea taken from the Dragon Book), but I don't know what I will do if somewhere in the process I have to go back and read a previous token.</li>
  </ul>
  
  <p>What is the right way to do these things?</p>