GMail API returns unspeficied error at random moments
Question: I'm running a Python app on Google App Engine which regularly checks the
latest emails for multiple users. I've noticed that at random moments, the API
returns the following error:

    
    
    error: An error occured while connecting to the server: Unable to fetch URL: https://www.googleapis.com/discovery/v1/apis/gmail/v1/rest?userIp=0.1.0.2
    

Typically, if the job runs again, it works. Below my code. I would like to
understand what is causing this, and more importantly how I can prevent the
error from blocking the process (e.g. how to run the job again with the same
values when this error is thrown).

    
    
    class getLatest(webapp2.RequestHandler):
      def post(self):
        try:
          email = self.request.get('email')
          g = Credentials.get_by_id(email)
          REFRESH_TOKEN = g.refresh_token
          start_history_id = g.hid
    
          credentials = OAuth2Credentials(None, settings.CLIENT_ID,
                                 settings.CLIENT_SECRET, REFRESH_TOKEN, None,
                                 GOOGLE_TOKEN_URI, None,
                                 revoke_uri=GOOGLE_REVOKE_URI,
                                 id_token=None,
                                 token_response=None)
    
          http = credentials.authorize(httplib2.Http())
          service = discovery.build("gmail", "v1", http=http)
          for n in range(0, 5): 
            try:
              history = service.users().history().list(userId=email, startHistoryId=start_history_id).execute(http=http)
              break
            except errors.HttpError, e:
              if n < 4:
                time.sleep((2 ** n) + random.randint(0, 1000) / 1000)
              else:
                raise
          changes = history['history'] if 'history' in history else []
          while 'nextPageToken' in history:
            page_token = history['nextPageToken']
            for n in range(0, 5): 
              try:
                history = service.users().history().list(userId=email, startHistoryId=start_history_id, pageToken=page_token).execute(http=http)
                break
              except errors.HttpError, e:
                if n < 4:
                  time.sleep((2 ** n) + random.randint(0, 1000) / 1000)
                else:
                  raise
            changes.extend(history['history'])
    
        except errors.HttpError, error:
            logging.exception('An error occurred: '+str(error))
            if error.resp.status == 401:
                # Credentials have been revoked.
                # TODO: Redirect the user to the authorization URL.
                raise NotImplementedError()
            else:
                stacktrace = traceback.format_exc()
                logging.exception('%s', stacktrace)
    

UPDATE

I updated the code based on the answer below, however it never seems to run
the request multiple times. The process just aborts as soon as an exception
takes place.


Answer: This error can occur due to multiple reasons including your agent going
offline for a moment to exceeding rate limit of Gmail API. You need to retry
in case of these temporary issues in a graceful manner. Below is the code edit
that may help:

    
    
    class getLatest(webapp2.RequestHandler):
      def post(self):
    
    try:
      email = self.request.get('email')
      g = Credentials.get_by_id(email)
      REFRESH_TOKEN = g.refresh_token
      start_history_id = g.hid
    
      credentials = OAuth2Credentials(None, settings.CLIENT_ID,
                             settings.CLIENT_SECRET, REFRESH_TOKEN, None,
                             GOOGLE_TOKEN_URI, None,
                             revoke_uri=GOOGLE_REVOKE_URI,
                             id_token=None,
                             token_response=None)
    
      http = credentials.authorize(httplib2.Http())
      service = discovery.build("gmail", "v1", http=http)
      for n in range(0, 5):  
      try:
          history = service.users().history().list(userId=email, startHistoryId=start_history_id).execute(http=http)
          break
      except Exception as e:
            # Apply exponential backoff.
            time.sleep((2 ** n) + random.randint(0, 1000) / 1000)
    
      changes = history['history'] if 'history' in history else []
      while 'nextPageToken' in history:
        page_token = history['nextPageToken']
    
        for n in range(0, 5):  
        try:
           history = service.users().history().list(userId=email, startHistoryId=start_history_id, pageToken=page_token).execute(http=http)
           changes.extend(history['history'])
           break
        except Exception as e:
            # Apply exponential backoff.
            time.sleep((2 ** n) + random.randint(0, 1000) / 1000)
    except errors.HttpError, error:
        logging.exception('An error occurred: '+str(error))
        if error.resp.status == 401:
            # Credentials have been revoked.
            # TODO: Redirect the user to the authorization URL.
            raise NotImplementedError()
        else:
            stacktrace = traceback.format_exc()
            logging.exception('%s', stacktrace)
    

Essentially this code retries if an exception occurs at google end from the
same point with exponential backoff. Add your error code, for which you want
to retry. Hope this resolves your problem.

