Efficient Way to Process Simple but Large Files in C++
<p>I'm working on a project that has me a bit over my head performance-wise.  I'm tasked with reading large (50MB or so) files of particle coordinates and displaying them.  I'd like to use C++ for this because I am learning it already.  </p>

<p>The coordinate structure in the files are simple, there's just alot (say a million or so):  </p>

<pre><code>1234.5667 5234.1566 //coordinate 1  
8532.6123 5152.6612 //coordinate 2  
....
</code></pre>

<p>Being a noob, I just want to read in the files line by line and store them in vectors, is this wrong?  Maybe I should be reading in the whole file first (buffered?), and then parsing the values?</p>

<p>Working example:</p>

<pre><code>clock_t c1 = clock();
vector&lt;double&gt; coords;
double coord;
ifstream fin("file.txt");
while(fin &gt;&gt; coord) {
    coords.push_back(coord);
}
cout &lt;&lt; "done. " &lt;&lt; coords.size()/2 &lt;&lt; " coords read.\n";
cout &lt;&lt; "took " &lt;&lt; (clock() - c1)/(double)CLOCKS_PER_SEC &lt;&lt; " seconds." &lt;&lt; endl;
</code></pre>

<p>And corresponding output on a 40MB file with 2 million coordinates:</p>

<pre><code>done. 2000000 coords read.
took 1.74 seconds.
</code></pre>

<p>Which is fast in my mind, but I'm thinking my mind isn't a good judge.</p>
