title: How can I sync data in S3 between a Beijing(China) bucket and a global one?
body: <p>Beijing (China) region is quite unique as almost everything of it is separate (from AWS Global). For instance, to use AWS CLI to list the objects we have to specify <code>region</code> and <code>endpoint-url</code>:</p>
  <pre class="lang-sh prettyprint-override"><code>aws --region cn-north-1 \
    --endpoint-url https://s3.cn-north-1.amazonaws.com.cn \
    --profile AN_AWS_CN_PROFILE \
    s3 ls s3://AN_AWS_CN_BUCKET/
  </code></pre>
  <p>My question is how can I sync data between a Beijing (China) bucket and a global one? To begin with, a global bucket is not recognised with the specified <code>region</code> &amp; <code>endpoint-url</code> above. For example,</p>
  <pre class="lang-sh prettyprint-override"><code>aws --region cn-north-1 \
    --endpoint-url https://s3.cn-north-1.amazonaws.com.cn \
    --profile AN_AWS_CN_PROFILE \
    s3 cp s3://AN_AWS_CN_BUCKET/ s3://AN_AWS_IRELAND_BUCKET/
  </code></pre>
  <p>will give</p>
  <blockquote>
  <p>fatal error: An error occurred (NoSuchBucket) when calling the ListObjects operation: The specified bucket does not exist</p>
  </blockquote>
  <p>(The destination bucket does exist.)</p>
  <p>So far I've explored CLI arguments, and <code>.aws/config</code> file to define profiles. Maybe it is possible to use multiple <code>profile</code> in a single command, but it doesn't seem possible to config <code>endpoint-url</code> in the <code>.aws/config</code>, as it is not mentioned <a href="http://docs.aws.amazon.com/cli/latest/topic/config-vars.html#cli-aws-help-config-vars" rel="nofollow noreferrer">here</a>.</p>
  <p>Now I just copy files to local then upload which would become an issue as the dataset grows quickly. Using EMR may resolve the scaling issue, but it adds an extra layer of complexity. I wonder if there's a better and possibly easy solution? Thank you.</p>