title: Debugging HXT performance problems
body: <p>I'm trying to use HXT to read in some big XML data files (hundreds of MB.)</p>
  
  <p>My code has a space-leak <em>somewhere</em>, but I can't seem to find it. I do have a little bit of a clue as to what is happening thanks to my very limited knowledge of the ghc profiling tool chain.</p>
  
  <p>Basically, the document is parsed, but not evaluated.</p>
  
  <p>Here's some code:</p>
  
  <pre><code>{-# LANGUAGE Arrows, NoMonomorphismRestriction #-}
  
  import Text.XML.HXT.Core
  import System.Environment (getArgs)
  import Control.Monad (liftM)
  
  main = do file &lt;- (liftM head getArgs) &gt;&gt;= parseTuba
            case file of(Left  m) -&gt; print "Failed."
                        (Right _) -&gt; print "Success."
  
  data Sentence t = Sentence [Node t] deriving Show
  data Node t = Word { wSurface :: !t } deriving Show
  
  parseTuba :: FilePath -&gt; IO (Either String ([Sentence String]))
  parseTuba f = do r &lt;- runX (readDocument [] f &gt;&gt;&gt; process)
                   case r of
                        []   -&gt; return $ Left "No parse result."
                        [pr] -&gt; return $ Right pr
                        _    -&gt; return $ Left "Ambiguous parse result!"
  
  process :: (ArrowXml a) =&gt; a XmlTree ([Sentence String])
  process = getChildren &gt;&gt;&gt; listA (tag "sentence" &gt;&gt;&gt; listA word &gt;&gt;&gt; arr (\ns -&gt; Sentence ns))
  
  word :: (ArrowXml a) =&gt; a XmlTree (Node String)
  word = tag "word" &gt;&gt;&gt; getAttrValue "form" &gt;&gt;&gt; arr (\s -&gt; Word s)
  
  -- | Gets the tag with the given name below the node.
  tag  :: (ArrowXml a) =&gt; String -&gt; a XmlTree XmlTree
  tag s = getChildren &gt;&gt;&gt; isElem &gt;&gt;&gt; hasName s
  </code></pre>
  
  <p>I'm trying to read a corpus file, and the structure is obviously something like <code>&lt;corpus&gt;&lt;sentence&gt;&lt;word form="Hello"/&gt;&lt;word form="world"/&gt;&lt;/sentence&gt;&lt;/corpus&gt;</code>.</p>
  
  <p>Even on the very small development corpus, the program takes ~15 secs to read it in, of which around 20% are GC time (that's way too much.)</p>
  
  <p>In particular, a lot of data is spending way too much time in DRAG state. This is the profile:</p>
  
  <p><img src="https://i.stack.imgur.com/PyEVq.png" alt="http://imgur.com/YC3jh"></p>
  
  <p>monitoring DRAG culprits. You can see that decodeDocument gets called a lot, and its data is then stalled until the very end of the execution.</p>
  
  <p>Now, I think this should be easily fixed by folding all this decodeDocument stuff into my data structures (<code>Sentence</code> and <code>Word</code>) and then the RT can forget about these thunks. The way it's currently happening though, is that the folding happens at the <em>very</em> <em>end</em> when I force evaluation by deconstruction of <code>Either</code> in the <code>IO</code> monad, where it could easily happen <em>online</em>. I see no reason for this, and my attempts to strictify the program have so far been in vain. I hope somebody can help me :-)</p>
  
  <p>I just can't even figure out too many places to put <code>seq</code>s and <code>$!</code>s in…</p>