title: Reading file with fread() in reverse order causes memory leak?
body: <p>I have a program that basically does this:</p>
  
  <ol>
  <li>Opens some binary file</li>
  <li>Reads the file <strong>backwards</strong> (by backwards, I mean it starts near EOF, and ends reading at beginning of file, i.e. reads the file "right-to-left"), using 4MB chunks</li>
  <li>Closes the file</li>
  </ol>
  
  <p>My question is: why memory consumption looks like below, even though there are no obvious memory leaks in my attached code?</p>
  
  <p><img src="https://i.stack.imgur.com/8EpxA.png" alt="Memory consumption during program execution"></p>
  
  <p>Here's the source of program that was run to obtain above image:</p>
  
  <pre class="lang-cpp prettyprint-override"><code>#include &lt;stdio.h&gt;
  #include &lt;string.h&gt;
  
  int main(void)
  {
      //allocate stuff
      const int bufferSize = 4*1024*1024;
      FILE *fileHandle = fopen("./input.txt", "rb");
      if (!fileHandle)
      {
          fprintf(stderr, "No file for you\n");
          return 1;
      }
      unsigned char *buffer = new unsigned char[bufferSize];
      if (!buffer)
      {
          fprintf(stderr, "No buffer for you\n");
          return 1;
      }
  
      //get file size. file can be BIG, hence the fseeko() and ftello()
      //instead of fseek() and ftell().
      fseeko(fileHandle, 0, SEEK_END);
      off_t totalSize = ftello(fileHandle);
      fseeko(fileHandle, 0, SEEK_SET);
  
      //read the file... in reverse order. This is important.
      for (off_t pos = totalSize - bufferSize, j = 0;
          pos &gt;= 0;
          pos -= bufferSize, j ++)
      {
          if (j % 10 == 0)
          {
              fprintf(stderr,
                  "reading like crazy: %lld / %lld\n",
                  pos, totalSize);
          }
  
          /*
           * below is the heart of the problem. see notes below
           */
          //seek to desired position
          fseeko(fileHandle, pos, SEEK_SET);
          //read the chunk
          fread(buffer, sizeof(unsigned char), bufferSize, fileHandle);
      }
  
      fclose(fileHandle);
      delete []buffer;
  }
  </code></pre>
  
  <p>I have also following observations:</p>
  
  <ol>
  <li><strong>Even though RAM usage jumps by 1GB, the whole program uses only 5MB thorough whole execution.</strong></li>
  <li>Commenting call to <code>fread()</code> out <strong>makes memory leak go away</strong>. This is weird, since I don't allocate anything anywhere near it, that could trigger memory leak...</li>
  <li>Also, reading the file normally instead of backwards (= commenting call to <code>fseeko()</code> out), <strong>makes memory leak go away as well</strong>. This is the <em>ultra-weird part</em>.</li>
  </ol>
  
  <p>Further information...</p>
  
  <ol>
  <li>Following doesn't help:
  
  <ol>
  <li>Checking results of <code>fread()</code> - yields nothing out of ordinary.</li>
  <li>Switching to normal, 32-bit <code>fseek</code> and <code>ftell</code>.</li>
  <li>Doing stuff like <code>setbuf(fileHandle, NULL)</code>.</li>
  <li>Doing stuff like <code>setvbuf(fileHandle, NULL, _IONBF, *any integer*)</code>.</li>
  </ol></li>
  <li>Compiled with g++ 4.5.3 on Windows 7 via cygwin and mingw; without any optimalizations, just <code>g++ test.cpp -o test</code>. Both present such behaviour.</li>
  <li>The file used in tests was 4GB long, full of zeros.</li>
  <li>The weird pause in the middle of the chart could be explained with some kind of temporary I/O hangup, unrelated to this question.</li>
  <li>Finally, if I wrap reading in infinite loop... the memory usage stops increasing after first iteration.</li>
  </ol>
  
  <hr>
  
  <p>I think it has to do with some kind of internal cache building up till it's filled with whole file. How does it really work behind the scenes? How can I prevent that in a portable way??</p>