Python-Requests full URL from error message
Question: I'm trying to unshorten URLs with the requests library. I'm currently doing
something like this:

    
    
    import requests
    from contextlib import closing
    
    def unshorten(url):
        session = requests.session()
        with closing(session.head(url)) as req:
            r = req
        if not r.headers.get('location'):  # not a redirect
            return url
        tmp_url = url
        try:
            for redir in session.resolve_redirects(r, r.request):
                if redir.status_code == 200 and not url_no_good(redir.url):  # ok!
                    return redir.url
                else:
                    tmp_url = redir.url
            else: # no acceptable responses :(
                return tmp_url
        except requests.exceptions.TooManyRedirects:
            return url
    

(url_no_good is shorthand for some tests to make sure the url isn't, e.g., a
DNS-mediated 404 page)

I'm running into a problem where a given url redirects to a no-longer-valid
site. I don't want the shortened link, I want the 'bad' url. I 'solved' this
with

    
    
    ERR_PAT = re.compile(r'host=\'([\w\d\.]+)\'')
    ...
            try:
                for redir in session.resolve_redirects(r, r.request):
                ...
            except requests.exceptions.TooManyRedirects:
                return url
            except requests.exceptions.ConnectionError as e:
                return 'http://' + re.search(ERR_PAT, e.message.message).group(1) + e.message.url
    

since, for a requests ConnectionError, `err.message.message` is the string
representation of the error and `err.message.url` is the non-domain part of
the url, e.g., `/foo/bar?baz=bloo`. Cobbling together the different parts of
the error message like this feels _incredibly_ hacky, and I'm really curious
if there's a less-involved way of handling this kind of thing.


Answer: When that exception is raised, `redir` is still bound to the last request that
you tried to follow the redirect on:

    
    
    try:
        redir = r  # in case the first redirect fails
        for redir in session.resolve_redirects(r, r.request):
            if redir.status_code == 200 and not url_no_good(redir.url):  # ok!
                return redir.url
            else:
                tmp_url = redir.url
        else: # no acceptable responses :(
            return tmp_url
    except requests.exceptions.TooManyRedirects:
        return url
    except requests.exceptions.ConnectionError:
        return redir.headers.get('location', url)
    

