title: How to iterate through BigQuery query results and write it to file
body: <p>I need to query the Google BigQuery table and export the results to gzipped file. 
  This is my current code. The requirement is that the each row data should be new line (\n) delemited. </p>
  
  <pre><code>def batch_job_handler(args):
  
      credentials = Credentials.from_service_account_info(service_account_info)
  
      client = Client(project=service_account_info.get("project_id"),
                      credentials=credentials)
  
      query_job = client.query(QUERY_STRING)
  
      results = query_job.result()  # Result's total_rows is 1300000 records
  
      with gzip.open("data/query_result.json.gz", "wb") as file:
          data = ""
          for res in results:
              data += json.dumps(dict(list(res.items()))) + "\n"
              break
          file.write(bytes(data, encoding="utf-8"))
  </code></pre>
  
  <p>The above solution works perfectly fine for small number of result but, gets too slow if result has 1300000 records. </p>
  
  <p>Is it because of this line: <code>json.dumps(dict(list(res.items()))) + "\n"</code> as I am constructing a huge string by concatenating each records by new line. </p>
  
  <p>As I am running this program in AWS batch, it is consuming too much time. I Need help on iterating over the result and writing to a file for millions of records in a faster way. </p>