title: Open and read thousands of files as fast as possible
body: <p>I need to open and read thousands of files as fast as possible.</p>
  
  <p>I have ran a few tests on 13 592 files and found Method 1 to be slightly faster than Method 2. These files are usually between 800 bytes and 4kB. I would like to know if there is anything I can do to make this I/O-bound process faster?</p>
  
  <pre><code>Method 1:
      Run 1: 3:05 (don't know what happened here)
      Run 2: 1:55
      Run 3: 2:06
      Run 4: 2:02
  Method 2:
      Run 1: 2:04
      Run 2: 2:08
      Run 3: 2:04
      Run 4: 2:12
  </code></pre>
  
  <p>Here's the code:</p>
  
  <pre><code>public class FileOpenerUtil
  {
  
      /// &lt;summary&gt;
      /// 
      /// &lt;/summary&gt;
      /// &lt;param name="fullFilePath"&gt;&lt;/param&gt;
      /// &lt;returns&gt;&lt;/returns&gt;
      public static string ReadFileToString(string fullFilePath)
      {
          while (true)
          {
              try
              {
                  //Methode 1
                  using (StreamReader sr = File.OpenText(fullFilePath))
                  {
                      string fullMessage = "";
                      string s;
                      while ((s = sr.ReadLine()) != null)
                      {
                          fullMessage += s + "\n";
                      }
                      return RemoveCarriageReturn(fullMessage);
                  }
                  //Methode 2
                  /*using (File.Open(fullFilePath, FileMode.Open, FileAccess.Read, FileShare.Read))
                  {
                      Console.WriteLine("Output file {0} ready.", fullFilePath);
                      string[] lines = File.ReadAllLines(fullFilePath);
                      //Every new line under the previous line
                      string fullMessage = lines.Aggregate("", (current, s) =&gt; current + s + "\n");
                      return RemoveCarriageReturn(fullMessage);
                      //ninject kernel
  
  
                  }*/
                  //Methode 3
  
              }
              catch (FileNotFoundException ex)
              {
                  Console.WriteLine("Output file {0} not yet ready ({1})", fullFilePath, ex.Message);
              }
              catch (IOException ex)
              {
                  Console.WriteLine("Output file {0} not yet ready ({1})", fullFilePath, ex.Message);
              }
              catch (UnauthorizedAccessException ex)
              {
                  Console.WriteLine("Output file {0} not yet ready ({1})", fullFilePath, ex.Message);
              }
          }
  
      }
  
      /// &lt;summary&gt;
      /// Verwijdert '\r' in een string sequence
      /// &lt;/summary&gt;
      /// &lt;param name="message"&gt;The text that has to be changed&lt;/param&gt;
      /// &lt;returns&gt;The changed text&lt;/returns&gt;
      private static string RemoveCarriageReturn(string message)
      {
          return message.Replace("\r", "");
      }
  }
  </code></pre>
  
  <p>The files I'm reading are .HL7 files and look like this:</p>
  
  <blockquote>
    <p>MSH|^~\&amp;|OAZIS||||20150430235954||ADT^A03|23669166|P|2.3||||||ASCII
    EVN|A03|20150430235954||||201504302359
    PID|1||6001144000||LastName^FirstName^^^Mevr.|LastName^FirstName|19600114|F|||GStreetName Number^^City^^PostalCode^B^H||09/3444556^^PH~0476519246echtg^^CP||NL|M||28783409^^^^VN|0000000000|60011402843||||||B||||N
    PD1||||003847^LastName^FirstName||||||||N|||0
    PV1|1|O|FDAG^000^053^001^0^2|NULL||FDAG^000^053^001|003847^LastName^FirstName||006813^LastName^FirstName|1900|00||||||006813^LastName^FirstName|0|28783409^^^^VN|1^20150430|01|||||||||||||||1|1||D|||||201504301336|201504302359
    OBX|1|CE|KIND_OF_DIS|RCM|1^1 Op medisch advies
    OBX|2|CE|DESTINATION_DIS|RCM|1^1 Terug naar huis</p>
  </blockquote>
  
  <p>Once I opened the file, I parse the string with <a href="https://github.com/j4jayant/hl7-cSharp-parser/tree/master/j4jayant.HL7.Parser" rel="nofollow">j4jayant's HL7 parser</a> and close the file.</p>