title: Efficient Way to Process Simple but Large Files in C++
body: <p>I'm working on a project that has me a bit over my head performance-wise.  I'm tasked with reading large (50MB or so) files of particle coordinates and displaying them.  I'd like to use C++ for this because I am learning it already.  </p>
  
  <p>The coordinate structure in the files are simple, there's just alot (say a million or so):  </p>
  
  <pre><code>1234.5667 5234.1566 //coordinate 1  
  8532.6123 5152.6612 //coordinate 2  
  ....
  </code></pre>
  
  <p>Being a noob, I just want to read in the files line by line and store them in vectors, is this wrong?  Maybe I should be reading in the whole file first (buffered?), and then parsing the values?</p>
  
  <p>Working example:</p>
  
  <pre><code>clock_t c1 = clock();
  vector&lt;double&gt; coords;
  double coord;
  ifstream fin("file.txt");
  while(fin &gt;&gt; coord) {
      coords.push_back(coord);
  }
  cout &lt;&lt; "done. " &lt;&lt; coords.size()/2 &lt;&lt; " coords read.\n";
  cout &lt;&lt; "took " &lt;&lt; (clock() - c1)/(double)CLOCKS_PER_SEC &lt;&lt; " seconds." &lt;&lt; endl;
  </code></pre>
  
  <p>And corresponding output on a 40MB file with 2 million coordinates:</p>
  
  <pre><code>done. 2000000 coords read.
  took 1.74 seconds.
  </code></pre>
  
  <p>Which is fast in my mind, but I'm thinking my mind isn't a good judge.</p>