title: C fread() and buffer management
body: <p>Let's suppose I am parsing a file of binary records.  The file could contain any number of records.  The format is:</p>
  
  <ul>
  <li>12 bytes C-string</li>
  <li>4 bytes length</li>
  <li>4 bytes offset</li>
  </ul>
  
  <p>I intend to use <code>fread()</code> to get records from the file, and transform them into an internal data structure.  As I understand it, <code>fread()</code> has its own internal buffer over the data file, to cut down on hard disk access for small reads.</p>
  
  <p>So there are two options:</p>
  
  <ul>
  <li>Loop: Read 20 bytes at a time with <code>fread()</code> and parse it.  This is easy, but has the drawback that I'm making a function call to <code>fread()</code> in every loop.  It's not as bad as a syscall, but it still incurs the overhead every 20 bytes.  In an extreme alternative case (say, one-byte-at-a-time file decompression) I could be doing a function call <em>every byte</em>.</li>
  <li>Read a big block at a time with <code>fread()</code>, then Loop: traverse my copy 20 bytes at a time, and parse it.  Parsing local memory is fast, but this has the drawback that I'm "buffering a buffer" - that is, filling the internal <code>fread()</code> buffer, just so I can turn around and <code>memcpy()</code> it to another structure.  Also, it's kind of tedious - either I have to fit the whole file in RAM myself, or I have to manage my own buffer-fill-and-flush routine to read the file in chunks.</li>
  </ul>
  
  <p>So I'm in a bit of a bind here, because neither solution seems theoretically optimal!</p>
  
  <p>My question is: <strong>Is there any pattern for using sequential file IO that avoids <em>both</em> of these drawbacks?</strong>  (And please, no "just profile it" comments - the question is not "which of these is better" but "is there a third alternative")</p>