Python urllib.request.urlopen() returning error 403
Question: I'm trying to download the HTML of a page (<http://www.guangxindai.com> in
this case) but I'm getting back an error 403. Here is my code:

    
    
    import urllib.request
    opener = urllib.request.build_opener()
    opener.addheaders = [('User-agent', 'Mozilla/5.0')]
    f = opener.open("http://www.guangxindai.com")
    f.read()
    

but I get error response.

    
    
    Traceback (most recent call last):
      File "<pyshell#7>", line 1, in <module>
        f = opener.open("http://www.guangxindai.com")
      File "C:\Python33\lib\urllib\request.py", line 475, in open
        response = meth(req, response)
      File "C:\Python33\lib\urllib\request.py", line 587, in http_response
        'http', request, response, code, msg, hdrs)
      File "C:\Python33\lib\urllib\request.py", line 513, in error
        return self._call_chain(*args)
      File "C:\Python33\lib\urllib\request.py", line 447, in _call_chain
        result = func(*args)
      File "C:\Python33\lib\urllib\request.py", line 595, in http_error_default
        raise HTTPError(req.full_url, code, msg, hdrs, fp)
    urllib.error.HTTPError: HTTP Error 403: Forbidden
    

I have tried different request headers, but still can not get correct
response. I can view the web through browser. It seems strange for me. I guess
the web use some method to block web spider. Does anyone know what is
happening? How can I get the HTML of page correctly?


Answer: If your aim is to read the html of the page you can use the following code. It
worked for me on Python 2.7

    
    
    import urllib
    f = urllib.urlopen("http://www.guangxindai.com")
    f.read()
    

