urllib2.URLError: <urlopen error unknown url type: c>
Question: I am using the below code to scrape over XFN content from web page
<http://ajaxian.com> but I am gatting the below error:

    
    
        Traceback (most recent call last):  File      "C:\Users\Somnath\workspace\us.chakra.social.web.microformat\src\microformats_xfn_scrape.py", line 40, in <module>
    page = urllib2.urlopen(URL)
      File "C:\Python27\lib\urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
      File "C:\Python27\lib\urllib2.py", line 394, in open
    response = self._open(req, data)
      File "C:\Python27\lib\urllib2.py", line 417, in _open
    'unknown_open', req)
      File "C:\Python27\lib\urllib2.py", line 372, in _call_chain
    result = func(*args)
      File "C:\Python27\lib\urllib2.py", line 1232, in unknown_open
    raise URLError('unknown url type: %s' % type)
    urllib2.URLError: <urlopen error unknown url type: c>
    

My code is as follows:

    
    
    '''
    Created on Jan 11, 2013
    
    @author: Somnath
    '''
    # Scraping XFN content from a web page
    # -*-coding: utf-8 -*-
    
    import sys
    import urllib2
    import HTMLParser
    from BeautifulSoup import BeautifulSoup
    
    # Try http://ajaxian.com
    URL = sys.argv[0]
    
    XFN_TAGS = set([
                'colleague',
                'sweetheart',
                'parent',
                'co-resident',
                'co-worker',
                'muse',
                'neighbor',
                'sibling',
                'kin',
                'child',
                'date',
                'spouse',
                'me',
                'acquaintance',
                'met',
                'crush',
                'contact',
                'friend',
                ])
    
    
    #try:
    page = urllib2.urlopen(URL)
    #except urllib2.URLError:
    #    print 'Failed to fetch ' + item
    
    #try:
    soup = BeautifulSoup(page)
    #except HTMLParser.HTMLParseError:
    #    print 'Failed to parse ' + item
    
    anchorTags = soup.findAll('a')
    
    for a in anchorTags:
        if a.has_key('rel'):
            if len(set(a['rel'].split()) & XFN_TAGS) > 0:
                tags = a['rel'].split()
                print a.contents[0], a['href'], tags
    

I am running PyDev under Eclipse and is using Run As --> Python Run and set
the Runtime Configuration with argument "http://ajaxian.com/". Can anybody
suggest where I am getting wrong?

One more thing: I have commented the two try blocks in my code because it was
giving an error undefined variable : item. If I want to re-include the try-
except blocks, should I give a blank definition of variable, item outside the
try blocks? How can I get rid of that problem?


Answer: As you suggested `sys.argv[0]` prints the path of your script, that is because
you call your script like

    
    
    python microformats_xfn‌​_scrape.py <some_argument>
    

and here index 0 of sys.argv is the name of the script and not the argument.

What you need to do is call your script with the `<url>` parameter, like :

    
    
    python microformats_xfn‌​_scrape.py http://www.ajaxian.com/
    

and in your script change `sys.argv[0]` to `sys.argv[1]` as url arguments
index is 1.

