How to get the content from a certain <table> using python?
Question: I have some `<tr>`s, like this:

    
    
    <tr align=center><td>10876151</td><td><a href=userstatus?user_id=yangfanhit>yangfanhit</a></td><td><a href=problem?id=3155>3155</a></td><td><font color=blue>Accepted</font></td><td>344K</td><td>219MS</td><td>C++</td><td>3940B</td><td>2012-10-02 16:42:45</td></tr>
    <tr align=center><td>10876150</td><td><a href=userstatus?user_id=BandBandRock>BandBandRock</a></td><td><a href=problem?id=2503>2503</a></td><td><font color=blue>Accepted</font></td><td>16348K</td><td>2750MS</td><td>G++</td><td>840B</td><td>2012-10-02 16:42:25</td></tr>
    

I want to fetch the content without html tags, like:

    
    
    yangfanhit
    3155
    Accepted
    344K
    219MS
    C++
    3940B
    2012-10-02 16:42:45
    

Now I'm using the following code to deal with it:

    
    
    response = urllib2.urlopen('http://poj.org/status', timeout=10)
    html = response.read()
    response.close()
    
    pattern = re.compile(r'<tr align.*</tr>')
    match = pattern.findall(html)
    pat = re.compile(r'<td>.*?</td>')
    p = re.compile(r'<[/]?.*?>')
    for item in match:
        for i in pat.findall(item):
            print p.sub(r'', i)
        print '================================================='
    

I'm new to regex and also new to python. So could you suggest some better
methods to process it?


Answer: You could use
[`BeautifulSoup`](http://www.crummy.com/software/BeautifulSoup/bs4/doc/) to
parse the html. To write the content of the table in csv format:

    
    
    #!/usr/bin/env python
    import csv
    import sys
    import urllib2
    from bs4 import BeautifulSoup # $ pip install beautifulsoup4
    
    soup = BeautifulSoup(urllib2.urlopen('http://poj.org/status'))
    
    writer = csv.writer(sys.stdout)
    for tr in soup.find('table', 'a')('tr'):
        writer.writerow([td.get_text() for td in tr('td')])
    

### Output

    
    
    Run ID,User,Problem,Result,Memory,Time,Language,Code Length,Submit Time
    10876151,yangfanhit,3155,Accepted,344K,219MS,C++,3940B,2012-10-02 16:42:45
    10876150,BandBandRock,2503,Accepted,16348K,2750MS,G++,840B,2012-10-02 16:42:25
    

