title: How to write a (shell) lexer by hand
body: <p>I'm working on a shell, a small bash-like shell, without scripting (if while ...)
  I have to make the lexer/parser (LL) by hand.</p>
  
  <p>So the lexer will transform the command (<strong>char *cmd</strong>) to a linked list (<strong>t_list *list</strong>).
  And the LL parser will transform the linked list (<strong>t_list *list</strong>) to an AST (binary tree <strong>t_btree *root</strong>) with a <a href="http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_10" rel="noreferrer">grammar</a></p>
  
  <p>So, I know how to make the LL parser but I don't know how to tokenize my command.</p>
  
  <p>For example: <code>ps | grep ls &gt;&gt; file ; make &amp;&amp; ./a.out</code></p>
  
  <p>=> <code>'ps' '|' 'grep' 'ls' '&gt;&gt;' 'file' ';' ''make '&amp;&amp;' './a.out'</code></p>
  
  <p>Thanks.</p>
  
  <p>(I don't wanna use any generator)</p>