title: Timeout Error When Uploading File from Google App Engine Storing to Google Cloud Storage
body: <p>I am uploading a file that is >10MB from App Engine to Google Cloud Storage via the code below.</p>
  
  <pre><code>gcs.bucket(bucket_name).blob(blob_name=file_path).upload_from_string(data, content_type=content_type)
  </code></pre>
  
  <p>I am using the GCS Python Client Library and not the built-in App Engine library because I am composing multiple >10MB files into a single file in Cloud Storage when the process is complete.</p>
  
  <p>The code is running in a task and has 10 minutes to get the data and upload the information as a CSV to GCS. The data is retrieved and converted into a CSV formatted string in less than 3 minutes. The code then tries to uploading the data to GCS, Stackdriver logging stops receiving logs and I wait ~10 minutes at which point I receive a flood of logs in Stackdriver up to the point of failure with the failure being:</p>
  
  <blockquote>
    <p>DeadlineExceededError: The overall deadline for responding to the HTTP request was exceeded.</p>
  </blockquote>
  
  <p>This issue is frustrating because of 2 things.</p>
  
  <ol>
  <li>This error is intermittent.</li>
  <li><p>Once 1 file succeeds they all succeed in seconds.</p>
  
  <ol>
  <li><p>During initial develop the issue never occurred. Only recently has the issue started to appear and is becoming more frequent.</p></li>
  <li><p>The <strong>first</strong> >10MB file always takes minutes to fail or succeed. Fails after 10 minutes, but may take anywhere from 1 to 9 <strong>minutes</strong> and then succeed. Once a file succeeds all future uploads of >10MB files take ~5-10 <strong>seconds</strong>.</p></li>
  </ol></li>
  </ol>
  
  <p>My theory is that there is some service that App Engine is using to upload the files to Google Cloud Storage that automatically goes to sleep after a certain time of no usage. When the service is asleep it takes a very long time to wake it back up. Once the service is awake it can upload to GCS without any issues, very quickly.</p>
  
  <p>Has anyone else run into this or have ideas on how to solve it?</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>Full error:</p>
  
  <p><code>(/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py:279)
  Traceback (most recent call last):
    File "/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 267, in Handle
      result = handler(dict(self._environ), self._StartResponse)
    File "/base/data/home/apps/s~pg-gx-n-app-200716/worker:20181030t154529.413639922318911836/lib/flask/app.py", line 2309, in __call__
      return self.wsgi_app(environ, start_response)
    File "/base/data/home/apps/s~pg-gx-n-app-200716/worker:20181030t154529.413639922318911836/lib/flask/app.py", line 2292, in wsgi_app
      response = self.full_dispatch_request()
    File "/base/data/home/apps/s~pg-gx-n-app-200716/worker:20181030t154529.413639922318911836/lib/flask/app.py", line 1813, in full_dispatch_request
      rv = self.dispatch_request()
    File "/base/data/home/apps/s~pg-gx-n-app-200716/worker:20181030t154529.413639922318911836/lib/flask/app.py", line 1799, in dispatch_request
      return self.view_functions[rule.endpoint](**req.view_args)
    File "/base/data/home/apps/s~pg-gx-n-app-200716/worker:20181030t154529.413639922318911836/worker.py", line 277, in cache_records
      cache_module.save_records(records=records, report_fields=report.report_fields, report_id=report.report_id, header=header_flag)
    File "/base/data/home/apps/s~pg-gx-n-app-200716/worker:20181030t154529.413639922318911836/storage/user/user.py", line 110, in save_records
      user_entry = User.__generate_user_csv(user=user, report_fields=report_fields)
    File "/base/data/home/apps/s~pg-gx-n-app-200716/worker:20181030t154529.413639922318911836/storage/user/user.py", line 55, in __generate_user_csv
      for index, attrib in enumerate(report_fields):
  DeadlineExceededError: The overall deadline for responding to the HTTP request was exceeded.</code></p>