Python catch timeout and repeat request
Question: I'm trying to use the Xively API with python to update a datastream but
occasionally I get a 504 error which seems to end my script.

How can I catch that error and more importantly delay and try again so the
script can keep going and upload my data a minute or so later?

Here's the block where I'm doing the uploading.

    
    
        # Upload to Xivity
        api = xively.XivelyAPIClient("[MY_API_KEY")
        feed = api.feeds.get([MY_DATASTREAM_ID])
        now = datetime.datetime.utcnow()
        feed.datastreams = [xively.Datastream(id='temps', current_value=tempF, at=now)]
        feed.update()
    

And here's the error I see logged when my script fails:

    
    
    Traceback (most recent call last):
     File "C:\[My Path] \ [My_script].py", line 39, in <module>
       feed = api.feeds.get([MY_DATASTREAM_ID])
     File "C:\Python34\lib\site-packages\xively_python-0.1.0_rc2-py3.4.egg\xively\managers.py", >line 268, in get
       response.raise_for_status()
     File "C:\Python34\lib\site-packages\requests-2.3.0-py3.4.egg\requests\models.py", line 795, >in raise_for_status
       raise HTTPError(http_error_msg, response=self)
    requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out
    

Thanks,

P.S. I've replaced my personal info with [MY_INFO] but obviously the correct
data appears in my code.


Answer: I usually use a decorator for this:

    
    
    from functools import wraps
    from requests.exceptions import HTTPError
    import time
    
    def retry(func):
        """ Call `func` with a retry.
    
        If `func` raises an HTTPError, sleep for 5 seconds
        and then retry.
    
        """
        @wraps(func)
        def wrapper(*args, **kwargs):
            try:
                ret = func(*args, **kwargs)
            except HTTPError:
                time.sleep(5)
                ret = func(*args, **kwargs)
            return ret
        return wrapper
    

Or, if you want to retry more than once:

    
    
    def retry_multi(max_retries):
        """ Retry a function `max_retries` times. """
        def retry(func):
            @wraps(func)
            def wrapper(*args, **kwargs):
                num_retries = 0 
                while num_retries <= max_retries:
                    try:
                        ret = func(*args, **kwargs)
                        break
                    except HTTPError:
                        if num_retries == max_retries:
                            raise
                        num_retries += 1
                        time.sleep(5)
                return ret 
            return wrapper
        return retry
    

Then put your code in a function like this

    
    
    #@retry
    @retry_multi(5) # retry 5 times before giving up.
    def do_call():
        # Upload to Xivity
        api = xively.XivelyAPIClient("[MY_API_KEY")
        feed = api.feeds.get([MY_DATASTREAM_ID])
        now = datetime.datetime.utcnow()
        feed.datastreams = [xively.Datastream(id='temps', current_value=tempF, at=now)]
        feed.update()
    

