title: Loop through all BigQuery jobs with Python
body: <p>I'm using the Google Python API to work with BigQuery.</p>
  
  <p>I'm trying to use <code>jobs().list()</code> and <code>jobs().list_next()</code> to paginate through
  all the jobs in my project.  I'm using a generator with the following code:</p>
  
  <pre><code>request = service.jobs().list(projectId=project_id,
                                allUsers=True,
                                stateFilter="done",
                                )
                                # or maxResults=500) 
                                # or maxResults=1000) 
                                # or maxResults=64000)
  while request is not None:
      response = request.execute()
      for x in response["jobs"]:
          yield x
      request = service.jobs().list_next(request, response)
  </code></pre>
  
  <p>The problem is that depending on how I use <code>maxResults</code>, I get different job lists.</p>
  
  <ul>
  <li>Using no <code>maxResults</code> argument I see 9986 jobs.  </li>
  <li>Using <code>maxResults=500</code> I see 8596 jobs.</li>
  <li>Using <code>maxResults=1000</code> I see 6743 jobs.  </li>
  <li>Using <code>maxResults=64000</code> I see 6743 jobs.</li>
  </ul>
  
  <p>I'm expecting the number of jobs to be the same each time, so I'm not sure if 
  I'm using the API correctly.  </p>
  
  <p>What's the correct way to loop through all the jobs in a project?</p>
  
  <hr>
  
  <p><em>(updated Wed Aug 14 15:30:29 CDT 2013)</em></p>
  
  <p>Still trying to figure this out.  I ran the code @Michael Manoochehri kindly provided three times, using different <code>maxResults</code>. Various information about the number of jobs reported each time and how they relate to each other are below:</p>
  
  <pre><code>s1 -&gt; no maxResults
  s2 -&gt; maxResults=500
  s3 -&gt; maxResults=1000
  
  |s1| -&gt; 10112
  |s2| -&gt; 8579
  |s3| -&gt; 6556
  
  |s1 intersection s2| -&gt; 8578
  |s2 difference s1| -&gt; 1
  |s1 difference s2| -&gt; 1534
  
  |s1 intersection s3| -&gt; 6556
  |s3 difference s1| -&gt; 0
  |s1 difference s3| -&gt; 3556
  
  |s3 intersection s2| -&gt; 6398
  |s2 difference s3| -&gt; 2181
  |s3 difference s2| -&gt; 158
  </code></pre>
  
  <p>I still can't make sense of why I'm not seeing a consistent total number of jobs regardless of the <code>maxResults</code> used.</p>