title: How does hsync() work in HDFS?
body: <p>Apache's documentation says </p>
  
  <pre><code>Similar to posix fsync, flush out the data in client's user buffer 
  all the way to the disk device (but the disk may have it in its cache).
  </code></pre>
  
  <p>which is quite straightforward to understand but when I use it in my code I don't understand the reason for following behavior.  </p>
  
  <ol>
  <li>I'm calling hsync() after each write(). When I pump data slowly one event after the other, after writing the first event, when I do a <code>cat</code> or <code>tail</code> on that file, I'm seeing the event in file. But, after writing 2 or 3 more events, I'm not seeing them. I've to close the stream to see those data getting flushed in the file. </li>
  <li>When I pump data at a scale of 10 events at a time or 20 or more events at a time, I'm consistently seeing the data in the file without closing the stream</li>
  <li>Similar to scenario 1, but here instead of doing hsync() after each write, I've scheduled it as a timer task. Here also I'm seeing data in the file without closing the stream, but here number of calls of hsycn() is more than number of calls of write(). For instance if I've written 3 events, I'm seeing them in the file after 4th or 5th call of hsync(). </li>
  </ol>
  
  <p>Can anyone explain me the reason for this behavior? Hope my description is clear.</p>