Dealing with extremely large strings
<p>Hey all, I am writing a logviewer application.  These logs are between 100-200 megs (almost never larger than that) of pure text.  </p>

<p>My application will be not only used to make the reading of said logs easier, but to offer tons of options on parsing (filters/searches, ect).  Anywho, the files are on a network drive so I currently use FileStream and BufferedStream to read them in, which depending on the file takes about 15 seconds.</p>

<p>My first question is there a faster way to do this?  ReadLine made parsing easier, but clocked in at about 25 seconds.</p>

<p>Secondly, if the user is just playing around with one file alot (using different combos of filters and searches, whats the best way to deal with it in the applications memory? I mean I could just re-read it each time but thats silly.  I tried storing the whole file in a global StringBuilder which seemed to work, but I couldnt pass it on to other functions (OutOfMemoryException).</p>

<p>Whats the best way to deal with what ends up being a gigantic string? (Keep in mind no manipulations are being made, just pure reads/filter out what the user dosn't want to see.</p>

<p>Thanks guys!</p>
