title: How to download a large file from Google Cloud Storage using Java with checksum control
body: <p>I want to download large files from Google Cloud Storage using the google provided Java library <a href="http://googlecloudplatform.github.io/google-cloud-java/0.8.0/apidocs/index.html" rel="nofollow noreferrer">com.google.cloud.storage</a>. I have working code, but I still have one question and one major concern:</p>
  
  <p>My major concern is, when is the file content actually downloaded? During (references to the code below) <code>storage.get(blobId)</code>, during <code>blob.reader()</code> or during <code>reader.read(bytes)</code>? This gets very important when it comes to <strong>how to handle an invalid checksum</strong>, what do I need to do in order to actually trigger that the file is fetched over the network again?</p>
  
  <p>The simpler question is: Is there built in functionality to do md5 (or crc32c) check on the received file in the google library? Maybe I don't need to implement it on my own.</p>
  
  <p>Here is my method trying to download big files from Google Cloud Storage:</p>
  
  <pre><code>private static final int MAX_NUMBER_OF_TRIES = 3;
  public Path downloadFile(String storageFileName, String bucketName) throws IOException {
      // In my real code, this is a field populated in the constructor.
      Storage storage = Objects.requireNonNull(StorageOptions.getDefaultInstance().getService());
  
      BlobId blobId = BlobId.of(bucketName, storageFileName);
      Path outputFile = Paths.get(storageFileName.replaceAll("/", "-"));
      int retryCounter = 1;
      Blob blob;
      boolean checksumOk;
      MessageDigest messageDigest;
      try {
          messageDigest = MessageDigest.getInstance("MD5");
      } catch (NoSuchAlgorithmException ex) {
          throw new RuntimeException(ex);
      }
  
      do {
          LOGGER.debug("Start download file {} from bucket {} to Content Store (try {})", storageFileName, bucketName, retryCounter);
          blob = storage.get(blobId);
          if (null == blob) {
              throw new CloudStorageCommunicationException("Failed to download file after " + retryCounter + " tries.");
          }
          if (Files.exists(outputFile)) {
              Files.delete(outputFile);
          }
          try (ReadChannel reader = blob.reader();
               FileChannel channel = new FileOutputStream(outputFile.toFile(), true).getChannel()) {
              ByteBuffer bytes = ByteBuffer.allocate(128 * 1024);
              int bytesRead = reader.read(bytes);
              while (bytesRead &gt; 0) {
                  bytes.flip();
                  messageDigest.update(bytes.array(), 0, bytesRead);
                  channel.write(bytes);
                  bytes.clear();
                  bytesRead = reader.read(bytes);
              }
          }
          String checksum = Base64.encodeBase64String(messageDigest.digest());
          checksumOk = checksum.equals(blob.getMd5());
          if (!checksumOk) {
              Files.delete(outputFile);
              messageDigest.reset();
          }
      } while (++retryCounter &lt;= MAX_NUMBER_OF_TRIES &amp;&amp; !checksumOk);
      if (!checksumOk) {
          throw new CloudStorageCommunicationException("Failed to download file after " + MAX_NUMBER_OF_TRIES + " tries.");
      }
      return outputFile;
  }
  </code></pre>