title: s3 concurrent writes
body: <p>I think I'm having a problem with concurrent s3 writes. Two (or more) processes are writing <em>almost</em> the same content to the same s3 location at the same time. I'd like to determine the concurrency rules that govern how this situation will play out. </p>
  
  <p>By design, all of the processes but one will get killed while writing to s3. (I had said they are writing "almost" the same content because all but one of the processes are getting killed. If all processes were allowed to live, they would end up writing the same exact content.)</p>
  
  <p>My theory is that the process getting killed is leaving an incomplete file on s3, and the other file (which presumably was written fully) is not being chosen as the one that gets to live on s3. I'd like to prove or disprove this theory. (I'm trying to find out if the issues are caused by concurrency issues during write to s3, or some other time).</p>
  
  <p>From the FAQ at <a href="http://aws.amazon.com/s3/faqs/" rel="noreferrer">http://aws.amazon.com/s3/faqs/</a> :</p>
  
  <blockquote>
    <p>Q: What data consistency model does Amazon S3 employ?</p>
    
    <p>Amazon S3 buckets in the US West (Oregon), US West (Northern
    California), EU (Ireland), Asia Pacific (Singapore), Asia Pacific
    (Tokyo), Asia Pacific (Sydney) and South America (Sao Paulo) Regions
    provide read-after-write consistency for PUTS of new objects and
    eventual consistency for overwrite PUTS and DELETES. Amazon S3 buckets
    in the US Standard Region provide eventual consistency.</p>
  </blockquote>
  
  <p>I'm using the US Standard Region. </p>
  
  <ul>
  <li>What does this answer say about concurrent writes? I think I understand the difference between "read-after-write consistency" vs "eventual consistency", but only in the context of what one sees when <em>reading</em> the object just after the write completes.</li>
  <li>Is it possible for the killed process to "win" and therefore end up with an incomplete file on s3? Or does s3 somehow ensure that the file will only get placed on s3 if the whole PUT operation completes?</li>
  <li>How does s3 decide which file "wins"? This is the real question here.</li>
  </ul>