title: How can I serve a .zip through an API Gateway with a Python Lambda?
body: <p><strong>Context:</strong> I have some log files in an S3 bucket that I need to retrieve. The permissions on the bucket prevent me from downloading them directly from the S3 bucket console. I need a "backdoor" approach to retrieve the files. I have an API Gateway setup to hit a Lambda, which will figure out what files to retrieve and get them from the S3 bucket. However, the files are over 10 MB and the AWS API Gateway has a maximum payload size of 10 MB. Now, I need a way to compress the files and serve them to the client as a downloadable zip.</p>
  
  <pre class="lang-py prettyprint-override"><code>import json
  import boto3
  import zipfile
  import zlib
  import os
  
  S3 = boto3.resource('s3')
  BUCKET = S3.Bucket(name="my-bucket")
  
  TEN_MEGA_BYTES = 1000000000
  
  def lambda_handler(event, context):
  
      # utilize Lambda's temporary storage (512 MB) 
      retrieved = zipfile.ZipFile("/tmp/retrieved.zip", mode="w", compression=zipfile.ZIP_DEFLATED, compresslevel=9)
  
      for bucket_obj in BUCKET.objects.all():
  
          # logic to decide which file I want is done here
  
          log_file_obj = BUCKET.Object(bucket_obj.key).get()
  
          # get the object's binary encoded content (a bytes object)
          content = log_file_obj["Body"].read()
  
          # write the content to a file within the zip
          # writestr() requires a bytes or str object
          retrieved.writestr(bucket_obj.key, content)
  
      # close the zip
      retrieved.close()
  
      # visually checking zip size for debugging
      zip_size = os.path.getsize("/tmp/retrieved.zip")
      print("{} bytes".format(zip_size), "{} percent of 10 MB".format(zip_size / TEN_MEGA_BYTES * 100))
  
      return {
          "header": {
              "contentType": "application/zip, application/octet-stream",
              "contentDisposition": "attachment, filename=retrieved.zip",
              "contentEncoding": "deflate"
          },
          "body": retrieved
      }
  
      # return retrieved
  </code></pre>
  
  <p>I have tried returning the zipfile object directly and within a JSON structure with headers that are supposed to be mapped from the integration response to the method response (i.e. the headers I'm setting programmatically in the Lambda should be mapped to the response headers that the client actually receives). In either case, I get a marshal error.</p>
  
  <pre><code>Response:
  {
      "errorMessage": "Unable to marshal response: &lt;zipfile.ZipFile [closed]&gt; is not JSON serializable",
      "errorType": "Runtime.MarshalError"
  }
  </code></pre>
  
  <p>I have done a lot of tinkering in the API Gateway in the AWS Console trying to set different combinations of headers and/or content types, but I am stuck. I'm not entirely sure what would be the correct header/content-type combination.</p>
  
  <p>From the above error message, it appears like Lambda can only return JSON structures but I am unable to confirm either way.</p>