title: Haskell/Parsec: how do I use Text.Parsec.Token with Text.Parsec.Indent (from the indents package)
body: <p>The <em>indents</em> package for Haskell's Parsec provides a way to parse indentation-style languages (like Haskell and Python). It redefines the <code>Parser</code> type, so how do you use the token parser functions exported by Parsec's <a href="http://hackage.haskell.org/packages/archive/parsec/3.1.3/doc/html/Text-Parsec-Token.html" rel="nofollow noreferrer"><code>Text.Parsec.Token</code></a> module, which are of the normal <code>Parser</code> type?</p>
  
  <h1>Background</h1>
  
  <ul>
  <li><a href="http://hackage.haskell.org/packages/archive/parsec/latest/doc/html/Text-Parsec.html" rel="nofollow noreferrer">Parsec</a> is a parser combinator library, whatever that means.</li>
  <li><a href="http://hackage.haskell.org/package/IndentParser-0.2.1" rel="nofollow noreferrer">IndentParser 0.2.1</a> is an old package providing the two modules <a href="http://hackage.haskell.org/packages/archive/IndentParser/0.2.1/doc/html/Text-ParserCombinators-Parsec-IndentParser.html" rel="nofollow noreferrer"><code>Text.ParserCombinators.Parsec.IndentParser</code></a> and <a href="http://hackage.haskell.org/packages/archive/IndentParser/0.2.1/doc/html/Text-ParserCombinators-Parsec-IndentParser-Token.html" rel="nofollow noreferrer"><code>Text.ParserCombinators.Parsec.IndentParser.Token</code></a></li>
  <li><a href="http://hackage.haskell.org/package/indents-0.3.3" rel="nofollow noreferrer">indents 0.3.3</a> is a new package providing the single module <a href="http://hackage.haskell.org/packages/archive/indents/0.3.3/doc/html/Text-Parsec-Indent.html" rel="nofollow noreferrer"><code>Text.Parsec.Indent</code></a></li>
  </ul>
  
  <p>Parsec comes with <a href="http://hackage.haskell.org/packages/archive/parsec/latest/doc/html/Text-Parsec.html" rel="nofollow noreferrer">a load of modules</a>. most of them export a bunch of useful parsers (e.g. <code>newline</code> from <a href="http://hackage.haskell.org/packages/archive/parsec/3.1.3/doc/html/Text-Parsec-Char.html#v:newline" rel="nofollow noreferrer"><code>Text.Parsec.Char</code></a>, which parses a newline) or parser combinators (e.g.  <code>count n p</code> from <a href="http://hackage.haskell.org/packages/archive/parsec/3.1.3/doc/html/Text-Parsec-Combinator.html#v:count" rel="nofollow noreferrer"><code>Text.Parsec.Combinator</code></a>, which runs the parser <em>p</em>, <em>n</em> times)</p>
  
  <p>However, the module <a href="http://hackage.haskell.org/packages/archive/parsec/3.1.3/doc/html/Text-Parsec-Token.html" rel="nofollow noreferrer"><code>Text.Parsec.Token</code></a> would like to export functions which are parametrized by the user with features of the language being parsed, so that, for example, the <code>braces p</code> function will run the parser <em>p</em> after parsing a '{' and before parsing a '}', ignoring things like comments, the syntax of which depends on your language.</p>
  
  <p>The way that <a href="http://hackage.haskell.org/packages/archive/parsec/3.1.3/doc/html/Text-Parsec-Token.html" rel="nofollow noreferrer"><code>Text.Parsec.Token</code></a> achieves this is that it exports a single function <a href="http://hackage.haskell.org/packages/archive/parsec/3.1.3/doc/html/Text-Parsec-Token.html#v:makeTokenParser" rel="nofollow noreferrer"><code>makeTokenParser</code></a>, which you call, giving it the parameters of your specific language (like what a comment looks like) and it returns a record containing all of the functions in <a href="http://hackage.haskell.org/packages/archive/parsec/3.1.3/doc/html/Text-Parsec-Token.html" rel="nofollow noreferrer"><code>Text.Parsec.Token</code></a>, adapted to your language as specified.</p>
  
  <p>Of course, in an indentation-style language, these would need to be adapted further (perhaps? here's where I'm not sure – I'll explain in a moment) so I note that <a href="http://hackage.haskell.org/package/IndentParser-0.2.1" rel="nofollow noreferrer">the (presumably obsolete) IndentParser package</a> provides a module <a href="http://hackage.haskell.org/packages/archive/IndentParser/0.2.1/doc/html/Text-ParserCombinators-Parsec-IndentParser-Token.html" rel="nofollow noreferrer"><code>Text.ParserCombinators.Parsec.IndentParser.Token</code></a> which looks to be a drop-in replacement for <a href="http://hackage.haskell.org/packages/archive/parsec/3.1.3/doc/html/Text-Parsec-Token.html" rel="nofollow noreferrer"><code>Text.Parsec.Token</code></a>.</p>
  
  <p>I should mention at some point that all the Parsec parsers are monadic functions, so they do magic things with state so that error messages can say at what line and column in the source file the error appeared</p>
  
  <h1>My Problem</h1>
  
  <p>For a couple of small reasons it appears to me that <a href="http://hackage.haskell.org/package/indents-0.3.3" rel="nofollow noreferrer">the indents package</a> is more-or-less the current version of IndentParser, however it does not provide a module that looks like <a href="http://hackage.haskell.org/packages/archive/IndentParser/0.2.1/doc/html/Text-ParserCombinators-Parsec-IndentParser-Token.html" rel="nofollow noreferrer"><code>Text.ParserCombinators.Parsec.IndentParser.Token</code></a>, it only provides <code>Text.Parsec.Indent</code>, so <strong>I am wondering how one goes about getting all the token parsers from <code>Text.Parsec.Token</code></strong> (like <code>reserved "something"</code> which parses the reserved keyword "something", or like <code>braces</code> which I mentioned earlier).</p>
  
  <p>It would appear to me that (the new) <code>Text.Parsec.Indent</code> works by some sort of monadic state magic to work out at what column bits of source code are, so that it doesn't need to modify the token parsers like <code>whiteSpace</code> from <a href="http://hackage.haskell.org/packages/archive/parsec/3.1.3/doc/html/Text-Parsec-Token.html#v:whiteSpace" rel="nofollow noreferrer"><code>Text.Parsec.Token</code></a>, which is probably why it doesn't provide a replacement module. But I am having a problem with types.</p>
  
  <p>You see, without <code>Text.Parsec.Indent</code>, all my parsers are of type <code>Parser Something</code> where Something is the return type and <code>Parser</code> is a type alias defined in Text.Parsec.String as</p>
  
  <pre><code>type Parser = Parsec String ()
  </code></pre>
  
  <p>but with <code>Text.Parsec.Indent</code>, instead of importing <code>Text.Parsec.String</code>, I use my own definition</p>
  
  <pre><code>type Parser a = IndentParser String () a
  </code></pre>
  
  <p>which makes all my parsers of type <code>IndentParser String () <em>Something</em></code>, where IndentParser is defined in Text.Parsec.Indent. but the token parsers that I'm getting from <code>makeTokenParser</code> in <code>Text.Parsec.Token</code> are of the wrong type.</p>
  
  <p>If this isn't making much sense by now, it's because I'm a bit lost. The type issue is <a href="http://spin.atomicobject.com/2012/03/16/using-text-parsec-indent-to-parse-an-indentation-sensitive-language-with-haskells-parsec-library/" rel="nofollow noreferrer">discussed a bit here</a>.</p>
  
  <hr>
  
  <p>The error I'm getting is that I've tried replacing the one definition of <code>Parser</code> above with the other, but then when I try to use one of the token parsers from <code>Text.Parsec.Token</code>, I get the compile error</p>
  
  <pre><code>Couldn't match expected type `Control.Monad.Trans.State.Lazy.State
                                  Text.Parsec.Pos.SourcePos'
              with actual type `Data.Functor.Identity.Identity'
  Expected type: P.GenTokenParser
                   String
                   ()
                   (Control.Monad.Trans.State.Lazy.State Text.Parsec.Pos.SourcePos)
    Actual type: P.TokenParser ()
  </code></pre>
  
  <h2>Links</h2>
  
  <ul>
  <li><a href="http://hackage.haskell.org/packages/archive/parsec/latest/doc/html/Text-Parsec.html" rel="nofollow noreferrer">Parsec</a></li>
  <li><a href="http://hackage.haskell.org/package/IndentParser-0.2.1" rel="nofollow noreferrer">IndentParser</a> (old package)</li>
  <li><a href="http://hackage.haskell.org/package/indents-0.3.3" rel="nofollow noreferrer">indents</a>, providing Text.Parsec.Indent (new package)</li>
  <li><a href="http://spin.atomicobject.com/2012/03/16/using-text-parsec-indent-to-parse-an-indentation-sensitive-language-with-haskells-parsec-library/" rel="nofollow noreferrer">some discussion of Parser types</a> with example code</li>
  <li><a href="http://changaco.net/blog/parse-indented-tree/" rel="nofollow noreferrer">another example of using Text.Parsec.Indent</a></li>
  </ul>
  
  <p>Sadly, neither of the examples above use token parsers like those in Text.Parsec.Token.</p>