title: C# and long SQL Server queries, what's the correct way to do this?
body: <p>My problem is simple in nature- query a database, and write output to a set of files (~1000).</p>
  
  <p>I have a SQL query I need to execute from C# on SQL Server. The query is fairly simple except for one of the values in an IN clause.</p>
  
  <p>As an example, the query may look like:</p>
  
  <pre><code>SELECT 
      ID, Name
  FROM
      MyTable
  WHERE
      SomeValue IN (/* list of values */)
  </code></pre>
  
  <p>The list of values is around 300 strings that are 6 characters each.  The values will ultimately look something like ("A00001", "A00002", etc), and are specified as input by the user of the application.</p>
  
  <p>My initial thought was to build the query out as a long string, with each of the values aggregated together.  This doesn't seem like the proper approach however, and seems to be error prone.  My next thought was to use a stored procedure, but I'm unsure of how the data should be passed into it.  How should this query be built in an efficient way where the data can be reliably passed in?</p>
  
  <p>My other option is to read in ALL data from SQL Server and then filter the results in C# as I receive them.  If I do this, however, I'll get around 9 million records which seems to be too many to read into memory as a data set.  Because I'm writing files after parsing ~5000 records, I don't want to keep a data reader connected to the database while I chew through the data I'm receiving.</p>
  
  <p>What is the best approach?</p>