title: Fetching data from large BigQuery table in python
body: <p>What I have is a BigQuery table(>5mil rows).</p>
  
  <p>I need to fetch this data in batches and process it inside AppEngine, python. </p>
  
  <p>The only way to fetch from a table that I know is to run SELECT query on this table and then iterate the result using tokens fetch_data returns.</p>
  
  <p>It looks like this:</p>
  
  <pre><code>query = u"""\
      SELECT url FROM %s
      """ % (query_table)
  
  query_job = client.run_async_query(str(uuid.uuid4()), query)
  
  query_job.begin()
  
  wait_for_job(query_job, 1)
  
  query_results = query_job.results()
  
  rows, total_rows, next_token = query_results.fetch_data(max_results=per_page, page_token=page_token)
  </code></pre>
  
  <p>This works on smaller tables, but on larger ones like mine it asks to allow large requests and specify target table. But this makes no sense to me. For to simply fetch data from a table I have to copy it to another table?</p>