title: More efficient use of aws s3 sync?
body: <p>Lately, we've noticed that our AWS bill has been higher than usual.  It's due to adding an <code>aws s3 sync</code> task to our regular build process.  The build process generates something around 3,000 files.  After the build, we run <code>aws s3 sync</code> to upload them <em>en masse</em> into a bucket.  The problem is that this is monetarily expensive. Each upload is costing us a ~$2 (we think) and this adds up to a monthly bill that raises the eyebrow.</p>
  
  <p>All but maybe 1 or 2 of those files actually change from build to build.  The rest are always the same.  Yet <code>aws s3 sync</code> sees that they all changed and uploads the whole lot.  </p>
  
  <p>The documentation says that <code>aws s3 sync</code> compares the file's last modified date and byte size to determine if it should upload.  The build server creates all those files brand-new every time, so the last modified date is always changed.</p>
  
  <p>What I'd like to do is get it to compute a checksum or a hash on each file and then use that hash to compare the files.  <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonResponseHeaders.html" rel="noreferrer">Amazon s3 already has the etag field which is can be an MD5 hash of the file.</a>  But the <code>aws s3 sync</code> command doesn't use etag. </p>
  
  <p>Is there a way to use etag? Is there some other way to do this?</p>
  
  <p>The end result is that I'd only like to upload the 1 or 2 files that are actually different (and save tremendous cost)</p>