title: C++ Optimal Block Size For Reading From A File
body: <p>I have a program that generates files containing random distributions of the character A - Z.  I have written a method that reads these files (and counts each character) using fread with different buffer sizes in an attempt to determine the optimal block size for reads.  Here is the method:</p>
  
  <pre><code>int get_histogram(FILE * fp, long *hist, int block_size, long *milliseconds, long *filelen)
  {
      char *buffer = new char[block_size];
      bzero(buffer, block_size);
  
      struct timeb t;
      ftime(&amp;t);
      long start_in_ms = t.time * 1000 + t.millitm;
  
      size_t bytes_read = 0;
      while (!feof(fp))
      {
          bytes_read += fread(buffer, 1, block_size, fp);
          if (ferror (fp))
          {
              return -1;
          }
          int i;
          for (i = 0; i &lt; block_size; i++)
          {
              int j;
              for (j = 0; j &lt; 26; j++)
              {
                  if (buffer[i] == 'A' + j)
                  {
                      hist[j]++;
                  }
              }
          }
      }
  
      ftime(&amp;t);
      long end_in_ms = t.time * 1000 + t.millitm;
      *milliseconds = end_in_ms - start_in_ms;
      *filelen = bytes_read;
  
      return 0;
  }
  </code></pre>
  
  <p>However, when I plot bytes/second vs. block size (buffer size) using block sizes of 2 - 2^20, I get an optimal block size of 4 bytes -- which just can't be correct.  Something must be wrong with my code but I can't find it.</p>
  
  <p>Any advice is appreciated.</p>
  
  <p>Regards.</p>
  
  <p>EDIT:</p>
  
  <p>The point of this exercise is to demonstrate the optimal buffer size by recording the read times (plus computation time) for different buffer sizes.  The file pointer is opened and closed by the calling code.</p>