title: Parsing a string in haskell
body: <p>I've got some strings that I want to parse into a list of "chunks". My strings look like this</p>
  
  <pre><code>"some text [[anchor]] some more text, [[another anchor]]. An isolated ["
  </code></pre>
  
  <p>And I expect to get back something like this</p>
  
  <pre><code>[
     TextChunk "some text ",
     Anchor "anchor",
     TextChunk " some more text, "
     Anchor "another anchor",
     TextChunk ". An isolated ["
  ]
  </code></pre>
  
  <p>I've managed to write a function and types that do what I need, but they seems overly ugly.
  Is there a nicer way to do this?</p>
  
  <pre class="lang-hs prettyprint-override"><code>data Token = TextChunk String | Anchor String deriving (Show)
  data TokenizerMode = EatString | EatAnchor deriving (Show)
  
  tokenize::[String] -&gt; [Token]
  tokenize xs =  
    let (_,_,tokens) = tokenize' (EatString, unlines xs, [TextChunk ""])
    in reverse tokens
  
  tokenize' :: (TokenizerMode, String, [Token]) -&gt; (TokenizerMode, String,[Token])
  -- If we're starting an anchor, add a new anchor and switch modes
  tokenize' (EatString, '[':'[':xs, tokens) = tokenize' (EatIdentifier, xs, (Identifier ""):tokens )
  -- If we're ending an anchor ass a new text chunk and switch modes
  tokenize' (EatAnchor, ']':']':xs, tokens) = tokenize' (EatString, xs, (TextChunk ""):tokens )
  -- Otherwise if we've got stuff to consume append it
  tokenize' (EatString, x:xs, (TextChunk t):tokens) = tokenize'( EatString, xs, (TextChunk (t++[x])):tokens)
  tokenize' (EatAnchor, x:xs, (Identifier t):tokens) = tokenize'( EatAnchor, xs, (Identifier (t++[x])):tokens)
  --If we've got nothing more to consume we're done.
  tokenize' (EatString, [], tokens) = ( EatString, [], tokens)
  --We'll only get here if we're given an invalid string
  tokenize' xx = error ("Error parsing .. so far " ++ (show xx))
  </code></pre>