Scraping with Python?
Question: I'd like to grab all the index words and its definitions from
[here](http://pali.hum.ku.dk/cpd/search.html). Is it possible to scrape web
content with Python?

Firebug exploration shows the following URL returns my desirable contents
including both index and its definition as to 'a'.

    
    
    http://pali.hum.ku.dk/cgi-bin/cpd/pali?acti=xart&arid=14179&sphra=undefined
    

what are the modules used? Is there any tutorial available?

I do not know how many words indexed in the dictionary. I`m absolute beginner
in the programming.


Answer: You should use [urllib2](http://docs.python.org/library/urllib2.html) for
gettting the URL contents and
[BeautifulSoup](http://www.crummy.com/software/BeautifulSoup/) for parsing the
HTML/XML.

Example - retrieving all questions from the StackOverflow.com main page:

    
    
    import urllib2
    from BeautifulSoup import BeautifulSoup
    
    page = urllib2.urlopen("http://stackoverflow.com")
    soup = BeautifulSoup(page)
    
    for incident in soup('h3'):
        print [i.decode('utf8') for i in incident.contents]
        print
    

This code sample was adapted from the [BeautifulSoup
documentation](http://www.crummy.com/software/BeautifulSoup/documentation.html).

