title: How to concatenate a million files in Google Cloud Storage
body: <p>According to the <a href="https://cloud.google.com/storage/docs/gsutil/commands/compose" rel="nofollow noreferrer">documentation</a> for Google Cloud Storage, there are a few limitations on using <code>gsutil compose</code> (see below).</p>
  <p>Is there a more efficient way to combine a large number of files in the same bucket (~1 million)?</p>
  <p>If I understand correctly, I would have to join groups of 32, then keep doing that and joining again?</p>
  <blockquote>
  <p>Note that there is a limit (currently 32) to the number of components that can be composed in a single operation.</p>
  <p>There is a limit (currently 1024) to the total number of components for a given composite object. This means you can append to each object at most 1023 times.</p>
  <p>There is a per-project rate limit (currently 200) to the number of components you can compose per second. This rate counts both the components being appended to a composite object as well as the components being copied when the composite object of which they are a part is copied.</p>
  </blockquote>