title: Atomicity of `write(2)` to a local filesystem
body: <p>Apparently POSIX states that</p>
  
  <blockquote>
    <p>Either a file descriptor or a stream is called a "handle" on the
    open file description to which it refers; an open file description
    may have several handles. […] All activity by the application
    affecting the file offset on the first handle shall be suspended
    until it again becomes the active file handle. […] The handles need
    not be in the same process for these rules to apply.
    -- <a href="http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_05_01" rel="noreferrer" title="2.5.1 Interaction of File Descriptors and Standard I/O Streams">POSIX.1-2008</a></p>
  </blockquote>
  
  <p>and</p>
  
  <blockquote>
    <p>If two threads each call [the write() function], each call shall
    either see all of the specified effects of the other call, or none
    of them.
    -- <a href="http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_09_07" rel="noreferrer" title="2.9.7 Thread Interactions with Regular File Operations">POSIX.1-2008</a></p>
  </blockquote>
  
  <p>My understanding of this is that when the first process issues a
  <code>write(handle, data1, size1)</code> and the second process issues
  <code>write(handle, data2, size2)</code>, the writes can occur in any order but
  the <code>data1</code> and <code>data2</code> <em>must</em> be both pristine and contiguous.</p>
  
  <p>But running the following code gives me unexpected results.</p>
  
  <pre class="lang-c prettyprint-override"><code>#include &lt;errno.h&gt;
  #include &lt;stdio.h&gt;
  #include &lt;stdlib.h&gt;
  #include &lt;string.h&gt;
  #include &lt;fcntl.h&gt;
  #include &lt;unistd.h&gt;
  #include &lt;sys/wait.h&gt;
  die(char *s)
  {
    perror(s);
    abort();
  }
  
  main()
  {
    unsigned char buffer[3];
    char *filename = "/tmp/atomic-write.log";
    int fd, i, j;
    pid_t pid;
    unlink(filename);
    /* XXX Adding O_APPEND to the flags cures it. Why? */
    fd = open(filename, O_CREAT|O_WRONLY/*|O_APPEND*/, 0644);
    if (fd &lt; 0)
      die("open failed");
    for (i = 0; i &lt; 10; i++) {
      pid = fork();
      if (pid &lt; 0)
        die("fork failed");
      else if (! pid) {
        j = 3 + i % (sizeof(buffer) - 2);
        memset(buffer, i % 26 + 'A', sizeof(buffer));
        buffer[0] = '-';
        buffer[j - 1] = '\n';
        for (i = 0; i &lt; 1000; i++)
          if (write(fd, buffer, j) != j)
            die("write failed");
        exit(0);
      }
    }
    while (wait(NULL) != -1)
      /* NOOP */;
    exit(0);
  }
  </code></pre>
  
  <p>I tried running this on Linux and Mac OS X 10.7.4 and using <code>grep -a 
  '^[^-]\|^..*-' /tmp/atomic-write.log</code> shows that some writes are not
  contiguous or overlap (Linux) or plain corrupted (Mac OS X).</p>
  
  <p>Adding the flag <code>O_APPEND</code> in the <code>open(2)</code> call fixes this
  problem. Nice, but I do not understand why. POSIX says</p>
  
  <blockquote>
    <p>O_APPEND
       If set, the file offset shall be set to the end of the file prior to each write.</p>
  </blockquote>
  
  <p>but this is not the problem here. My sample program never does
  <code>lseek(2)</code> but share the same file description and thus same file
  offset.</p>
  
  <p>I have already read similar questions on Stackoverflow but they still
  do not fully answer my question.</p>
  
  <p><a href="https://stackoverflow.com/questions/6896011/atomic-write-on-file-from-two-process">Atomic write on file from two process</a> does not specifically
  address the case where the processes share the same <em>file description</em>
  (as opposed to the same file).</p>
  
  <p><a href="https://stackoverflow.com/questions/7660293/how-does-one-programmatically-determine-if-write-system-call-is-atomic-on-a-pa">How does one programmatically determine if “write” system call is atomic on a particular file?</a> says that</p>
  
  <blockquote>
    <p>The <code>write</code> call as defined in POSIX has no atomicity guarantee at all.</p>
  </blockquote>
  
  <p>But as <a href="http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_09_07" rel="noreferrer" title="2.9.7 Thread Interactions with Regular File Operations">cited above</a> it does have some. And what’s more,
  <code>O_APPEND</code> seems to trigger this atomicity guarantee although it seems
  to me that this guarantee should be present even without <code>O_APPEND</code>.</p>
  
  <p>Can you explain further this behaviour ?</p>