which one is better to test a valid url in python
Question: I want to check whether a particular url exists or not.

I came across two methods.

url = "<http://www.google.com>"

1.

    
    
    import urllib2
    response = urllib2.urlopen(url)
    response.code  # check what is the response code
    

2.

    
    
    import httplib 
    conn = httplib.HTTPConnection(url) 
    conn.request('HEAD', '/') 
    response = conn.getresponse() 
    if response.status == 200: # check the status code
        # do something
    

Though both will solve my purpose, but which one is a better method to achieve
this purpose.

Thanks in advance for help.


Answer: You can try something along those lines, the point being, that you usually
also want to handle the errors you get, if there is a problem with fetching
the url.

    
    
    In [4]: import urllib2
    
    In [5]: def test(url):
       ...:     try:
       ...:         response = urllib2.urlopen(url)
       ...:     except urllib2.HTTPError as e:
       ...:         return e.code,None
       ...:     return response.code,response
    
    In [6]: test('http://www.google.com')
    Out[6]: 
    (200,
     <addinfourl at 154469068 whose fp = <socket._fileobject object at 0x92caa2c>>)
    
    In [7]: test('http://www.google.com/foobar')
    Out[7]: (404, None)
    

In reality you would also need to handle `urllib2.URLError`:

    
    
    In [10]: def test(url):
        ...:     try:
        ...:         response = urllib2.urlopen(url)
        ...:     except urllib2.HTTPError as err:
        ...:         return err.code, None
        ...:     except urllib2.URLError as err:
        ...:         return err.reason, None
        ...:     return response.code,response
    
    In [11]: test('http://www.google.foo')
    Out[11]: (socket.gaierror(-2, 'Name or service not known'), None)
    

