title: Why can 'dd' read from a pipe faster than my own program using ifstream?
body: <p>I have two programs that pass data to each other via linux pipes (named or otherwise).  I need to hit a transfer rate of ~2600 MB/s between the two programs, but am currently seeing a slower rate of about ~2200 MB/s.  However, I found that if I replace my 2nd process with 'dd' instead, the transfer rate jumps to over 3000 MB/s.  Is there something about the way my program is reading from the pipe that is less efficient than the way 'dd' does it?  What can I do to improve this throughput?  Is 'ifstream' inherently slower than other methods of reading binary data from pipe?</p>
  
  <p>To summarize the two scenarios:</p>
  
  <blockquote>
    <p><strong>Scenario 1:</strong></p>
    
    <p>Program 1 -> [named pipe] -> Program 2</p>
    
    <p><em>Yields ~2200 MB/s transfer rate</em></p>
    
    <p><strong>Scenario2:</strong></p>
    
    <p>Program 1 -> [named pipe] -> 'dd if=pipename of=/dev/null bs=8M'</p>
    
    <p><em>Yields ~3000 MB/s transfer</em>
    rate.</p>
  </blockquote>
  
  <p>Here is the way my Program 2 currently reads from pipe:</p>
  
  <pre><code>ifstream inputFile;
  inputFile.open(inputFileName.c_str(), ios::in | ios::binary);
  while (keepLooping)
  {
      inputFile.read(&amp;buffer[0], 8*1024*1024);
      bytesRead = inputFile.gcount();
      //Do something with data
  }
  </code></pre>
  
  <p><strong>Update:</strong></p>
  
  <p>I have now tried using 'read(fd, &amp;buffer[0], 8*1024*1024)' instead of istream, seemed to show a <em>mild</em> improvement (but not as much as dd)</p>
  
  <p>I also tried using stream->rdbuf()->sgetn(&amp;buffer[0], 8*1024*1024) instead of stream->read(), which did not help.</p>