title: best way to achieve 30,000 inserts per second into SQL
body: <p>Without using SSIS or Bulk Inserts, Bulk Load, or Bulk anything; I need to be able to just call a stored procedure that inserts data.</p>
  
  <p>I have a 2.8 gig file that contains over 18 million records.
  My solution, written in C# and taking advantage of Parallel processing and ASYNC-AWAIT, uses multithreaded processes on multiple servers to parse this file in 10 minutes. However, no matter what I try, additional threads, additional servers, even inserting multiple records in one call by passing a table valued parameter, the fastest I can insert these records into an SQL database is 3100 per second. The stored procedure contains only the INSERT statement, and the table has no indexes or keys. At this rate it takes 100 minutes to store the parsed results  into a database. I have a need to bring this down to 10 minutes, meaning I have to increase my ability to insert records ten-fold. The database is my only bottleneck.</p>
  
  <p>A) Will using multiple filegroups do the trick?</p>
  
  <p>B) What about breaking out the table into versions, such as:
  EtlRecords001, EtlRecords002, EtlRecords003, ..., EtlRecords100.
  The idea here being to make the tables smaller.</p>
  
  <p>C) Lastly, spreading the work by using multiple SQL servers
  The idea here also being to use smaller tables AND combined with using several database servers.</p>
  
  <p>In the case of B and C, the solution would obviously have processes on different machines target different; B.) table versions; C) server instances.</p>
  
  <p>Not knowing much about filegroups, I have to believe there's a better way than cases B and C.</p>
  
  <p>And lastly, buying a 48 core server with 128 gig of ram isn't a viable option for me :)</p>