Web-scraper for a larger program I have a web scraper that I use in a part of a larger program. However, I feel like I semi-repeat my code a lot and take up a lot of room. Is there any way I can condense this code?

def read_mail(mail):
        url = [mail] # Ignore this line, please.
        i = 0 # Ignore this line, please.
        droppedSource = '&lt;td class="item_dropped"&gt;(.+?)&lt;/td&gt;' # Gets whatever is inbetween the tags
        destroyedSource = '&lt;td class="item_destroyed"&gt;(.+?)&lt;/td&gt;'
        totalSource = '&lt;strong class="item_dropped"&gt;(.+?)&lt;/strong&gt;'

        droppedText = re.compile(droppedSource) # Converts regex string into something that can be interpreted by regular library
        destroyedText = re.compile(destroyedSource)
        totalText = re.compile(totalSource)

        html = urllib.urlopen(url[i]).read() # ignore the url[i] part of this line, please.

        dropped = re.findall(droppedText,html)
        destroyed = re.findall(destroyedText,html)
        total = re.findall(totalText,html)

        return("Info: " + str(dropped[0])+str(destroyed[0])+str(total[0]))
First of all I would recommend not to use regex for handling HTML. You can use a library like BeautifulSoup for this.
As all we're doing is finding the first match using tag name and class name we can define a function that uses BeautifulSoup to find such matches bases on the tag and class name. BeautifulSoup provides two functions find and findAll, find returns the first match and findAll returns all the matches.
Note that in regex just to find the first match you should not use re.findall, better use re.search which only returns the first match found otherwise None.
On the last return line we can use string formatting.




from BeautifulSoup import BeautifulSoup
from functools import partial


def find_by_tag_name_class(soup, tag, cls_name, many=False):
    if many:
        matches = soup.findAll(tag, {"class": cls_name})
        return [match.text for match in matches]
    else:
        match = soup.find(tag, {"class": cls_name})
        return match.text


def read_mail(html):
    soup = BeautifulSoup(html)
    # Instead of passing the same `soup` multiple times to 
    # `find_by_tag_name_class` we can create a partial function
    # with `soup` already applied to it.
    find_by_tag_name_class_soup = partial(find_by_tag_name_class, soup) 
    dropped = find_by_tag_name_class_soup('td', 'item_dropped')
    destroyed = find_by_tag_name_class_soup('td', 'item_destroyed')
    total = find_by_tag_name_class_soup('strong', 'item_dropped')

    return "Info: {} {} {} " .format(dropped, destroyed, total)

html = '''&lt;td class="item_dropped"&gt;Foo bar&lt;/td&gt;&lt;td class="item_dropped"&gt;spam eggs&lt;/td&gt;
&lt;td class="item_destroyed"&gt;Hakuna&lt;/td&gt;&lt;td class="item_destroyed"&gt;Matatat&lt;/td&gt;
&lt;strong class="item_dropped"&gt;Some strong text&lt;/strong&gt;&lt;strong class="item_dropped"&gt;Even more strong text&lt;/strong&gt;'''

print read_mail(html)
# Info: Foo bar Hakuna Some strong text 




Note that in the latest version of BeautifulSoup findAll has been renamed to find_all.