Python Automated Web Page downloading, with Username, Password and Cookies
Question: I'm trying to implement in Python a simple program that reads rom web pages
and writes them to files. There are about 2000 pages of messages incrementally
numbered, but some numbers are missing.

The Web site is username and password protected, and I'm using the same
username and password I normally use to access it manually. I'm using some
code examples with cookie handling I found in the official Python web site,
but when I try them the website I'm trying to copy replies

> "Your browser is not accepting our cookies. To view this page, please set
> your browser preferences to accept cookies. (Code 0)"

Obviously there is a problem with cookies, and perhaps I'm not handling
username and password correctly. Any suggestion regarding the following code?

    
    
    import urllib2
    import cookielib
    import string
    import urllib
    def cook():
        url="http://www.URL.com/message/"
        cj = cookielib.LWPCookieJar()
        authinfo = urllib2.HTTPBasicAuthHandler()
        realm = "http://www.URL.com"
        username = "ID"
        password = "PSWD"
        host = "http://www.URL.com/message/"
        authinfo.add_password(realm, host, username, password)
        opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj), authinfo)
        urllib2.install_opener(opener)
    
        # Create request object
        txheaders = { 'User-agent' : "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)" }
        try:
            req = urllib2.Request(url, None, txheaders)
            cj.add_cookie_header(req)
            f = urllib2.urlopen(req)
    
        except IOError, e:
            print "Failed to open", url
            if hasattr(e, 'code'):
                print "Error code:", e.code
    
        else:
    
            print f
    
    cook
    url="http://www.URL.com/message/"
    urllib.urlretrieve(url + '1', 'filename')
    


Answer: Take a look in [Bolacha](https://github.com/gabrielfalcao/bolacha), it's a
wrapper to httplib2 that handles cookies and other stuff...

