Python: How do you login to a page and view the resulting page in a browser?
Question: I've been googling around for quite some time now and can't seem to get this
to work. A lot of my searches have pointed me to finding similar problems but
they all seem to be related to cookie grabbing/storing. I think I've set that
up properly, but when I try to open the 'hidden' page, it keeps bringing me
back to the login page saying my session has expired.

    
    
    import urllib, urllib2, cookielib, webbrowser
    
    username = 'userhere'
    password = 'passwordhere'
    url = 'http://example.com'
    webbrowser.open(url, new=1, autoraise=1)
    cj = cookielib.CookieJar()
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
    login_data = urllib.urlencode({'username' : username, 'j_password' : password})
    opener.open('http://example.com', login_data)
    resp = opener.open('http://example.com/afterlogin')
    print resp
    webbrowser.open(url, new=1, autoraise=1)
    


Answer: First off, when doing cookie-based authentication, you need to have a
[`CookieJar`](http://docs.python.org/library/cookielib.html#cookielib.CookieJar)
to store your cookies in, much in the same way that your browser stores its
cookies a place where it can find them again.

After opening a login-page through python, and saving the cookie from a
successful login, you should use the
[`MozillaCookieJar`](http://docs.python.org/library/cookielib.html#cookielib.MozillaCookieJar)
to pass the python created cookies to a format a firefox browser can parse.
Firefox 3.x no longer uses the cookie format that MozillaCookieJar produces,
and I have not been able to find viable alternatives.

If all you need to do is to retrieve specific (in advance known format
formatted) data, then I suggest you keep all your HTTP interactions within
python. It is much easier, and you don't have to rely on specific browsers
being available. If it is absolutely necessary to show stuff in a browser, you
could render the so-called 'hidden' page through urllib2 (which incidentally
integrates very nicely with cookielib), save the html to a temporary file and
pass this to the
[`webbrowser.open`](http://docs.python.org/library/webbrowser.html?highlight=webbrowser#webbrowser.open)
which will then render that specific page. Further redirects are not possible.

