title: Reading large binary file backward use pread
body: <p>I am looking for the best performance reverse read from end of file for reading very large binary file (+100GB) in Linux. Currently I have tried to measure performance of <strong>pread</strong> for reading a 10GB file. And the test was on SSD not spin disk. File system on SSD is exFAT.</p>
  
  <p>If reading file forward (from low offset to high offset) the performance is great. In my test I read 48k 10 times the elapsed time ~ 3ms.
  If reading file from end of file and reading reversely (from high offset to low offset), the elapsed time increase to 33ms.</p>
  
  <p>I've tried to clean up cached by calling posix_fadvice(POSIX_FADV_DONTNEED) after every read. I also tried to set up O_DIRECT | O_NOATIME | O_LARGEFILE | O_NONBLOCK. But these method has no effect on the performance.</p>
  
  <p>Here's the code what I've tested:</p>
  
  <pre><code>fd = open(filename, O_RDONLY | O_DIRECT | O_NOATIME | O_LARGEFILE | O_NONBLOCK);
  for(long long int i=0x27ff7c000; i&lt;0x280000000; i+=0xc000)
  {    
     pread(fd, buffer, 0xc000, i);
     posix_fadvise(fd, i, 0xc000, POSIX_FADV_DONTNEED);
  }
  //measure evey read ~ 3ms
  
  for(long long int i=0x280000000; i&gt;0x27ff7c000; i-=0xc000)
  {    
     pread(fd, buffer, 0xc000, i);
     posix_fadvise(fd, i, 0xc000, POSIX_FADV_DONTNEED);
  }
  //measure every read ~ 33ms
  </code></pre>
  
  <p>When doing the forward read, I notice the elapsed time of the first reading
  is also longer. Does this mean when doing pread with large offset it takes long time to place file pointer to the according offset? and afterwards the file pointer just increased from current position? But if read reversely it needs to re-calculate file pointer from beginning of the file so it takes longer time? If so, when reading 100GB reversely would be much worst than 
  reading 10GB.</p>
  
  <p>Any suggestion to help improve reverse read is great appreciated.</p>