Python - Printing an HTML page give empty response for some sites
Question: I want to print a html page from a site (whoscored.com). I can print, but if I
try a sub-domain, gives an empty response:

    
    
    import urllib2
    htmlfile =urllib2.urlopen("http://whoscored.com/Matches/829663/Live/")
    html = htmlfile.read()    
    print html
    


Answer: First of all, yes, the page you've provided _does not exist_.

Also, you need to _provide the`User-Agent` header_ to get and see the actual
404 HTML response. Example using [`requests` library](http://docs.python-
requests.org/en/latest/):

    
    
    >>> import requests
    >>> 
    >>> headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 Safari/537.36'}
    >>> response = requests.get("http://whoscored.com/829652/Live/", headers=headers)
    >>> print response.content
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
    <html>
    <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8" /> 
    <meta http-equiv="content-language" content="en" />
    <title>WhoScored.com</title>
    </head>
    <body style="padding: 20px; font-family:Arial,Helvetica,sans-serif; background-color:#222222;">
        <div style="margin:0 auto; padding: 40px 20px; width:560px; background-color:#fff;">
            The page you requested does not exist in <a href="http://www.whoscored.com">WhoScored.com</a>
        </div>
    </body>
    </html>
    

