title: Optimising for O_DIRECT writes
body: <p>I'm trying to write an application which will need to write to disk very quickly. I've hit my performance target for writing to disk, which is great.</p>
  
  <p>However, I've noticed that writing to disk so quickly is using a lot of CPU time: One core is maxed out, another is at 80% and another 2 are 10-20%. So I heard that O_DIRECT can decrease CPU consumption by avoiding all of those copies into kernel space and then copies to disk.</p>
  
  <p>I ran a small test program which confirmed this - CPU usage drops to 50% of one core - much better.</p>
  
  <p>However, I never got quite the same throughput as I did when doing normal writes and to make it quick, I had to use a <em>really</em> big record size (something like 130MB!)</p>
  
  <p>So, the question is, I guess:</p>
  
  <ul>
  <li>Is there a better way to decrease CPU usage than O_DIRECT for writes?
  or</li>
  <li>How can I get a similar throughput to what the kernel gets?</li>
  </ul>
  
  <p>My enviroment is Linux, I'm using a RAID 50, and I'm able to buffer writes until I hit some optimal record size. There will be only one writer at a time.</p>