title: It is possible to stream a large SQL Server database result set using Dapper?
body: <p>I have about 500K rows I need to return from my database (please don't ask why).</p>
  
  <p>I will then need to save these results as XML (more URGH) and the ftp this file to somewhere magical.</p>
  
  <p>I also need to transform the each row in the result set. </p>
  
  <p>Right now, this is what I'm doing with say .. <code>TOP 100</code> results:</p>
  
  <ul>
  <li>using Dapper's <code>Query&lt;T&gt;</code> method, which throws the entire result set into memory </li>
  <li>I then use AutoMapper to convert the database POCO to my FileResult POCO </li>
  <li>Convert to XML</li>
  <li>Then save this collection to the file system</li>
  <li>Then FTP</li>
  </ul>
  
  <p>This works fine for 100 rows, but I get an Out Of Memory exception with AutoMapper when trying to convert the 500K results to a new collection.</p>
  
  <p>So, I was wondering if I could do this...</p>
  
  <ul>
  <li>Stream data from DB using Dapper</li>
  <li>For <em>each row</em>, automapper it</li>
  <li>Convert to XML</li>
  <li>Stream result to disk</li>
  <li>&lt;repeat for each row&gt;</li>
  <li>Now ftp that file to magic-land</li>
  </ul>
  
  <p>I'm trying to stop throwing <em>everything</em> into RAM. My thinking is that if I can stream stuff, it's more memory efficient as I only work on a single result set of data.</p>