Fetching data through HTTP and caching JSON result in a file The task is to fetch some JSON data from an API with a GET request. If the location is not available for any reason, read a cache file. Otherwise write the cache file for future use.

The following function works but it is clumsy due to:


The nested try/except, which is difficult to read
It's difficult to figure out what the error is (I am catching both HTTPError and the JSONDecodeError to avoid further nesting
I don't dare to use a context manager for opening the file (with) due to a further nesting level within a try/except clause


Have you got any ideas for improvement?

def fetch_list(location=MY_LOCATION, cache_file=CACHE_FILE):
    """Goes to the default location and returns
    a python list
    """
    http = urllib3.PoolManager()
    try:
        r = http.request("GET",
                         location)
        raw_data = r.data.decode("utf-8")
        data = json.loads(raw_data)
    except (urllib3.exceptions.HTTPError, JSONDecodeError):
        logger.error("Cannot access Intranet List Location - fetching cache")
        try:
            data = json.loads(open(cache_file).readlines())
        except (IOError, JSONDecodeError):
            logger.error("Cache File not found or broken")
            raise
    else:
        with open(cache_file, "w") as f:
            f.write(raw_data)

    return data
If your logger object is derived from the logging module, use the logger.exception which'll log the traceback information as well. The docs (linked above) also specify this behaviour:


  Exception info is added to the logging message. This function should
  only be called from an exception handler.


You can avoid nesting try-except block by just passing in first exception and place the write to cache inside the first try block itself:

def fetch_list(location=MY_LOCATION, cache_file=CACHE_FILE):
    """Goes to the default location and returns a python list.
    """
    try:
        http = urllib3.PoolManager()
        request = http.request("GET", location)
        raw_data = request.data.decode("utf-8")
        data = json.loads(raw_data)
        with open(cache_file, "w") as cache:
            cache.write(raw_data)
        return data
    except (urllib3.exceptions.HTTPError, JSONDecodeError) as exc:
        logger.exception("Cannot access Intranet List Location - fetching cache")
    try:
        data = json.loads(open(cache_file).readlines())
        return data
    except (IOError, JSONDecodeError):
        logger.exception("Cache File not found or broken")
        raise