Python Beautiful Soup Scraping Exact Content From Charts
Question: In python using beautiful soup I want to be able to grab specific text
`<a>/numbers<td>` from a sortable table online.

[http://www.nfl.com/stats/categorystats?archive=false&conference=null&role=OPP&offensiveStatisticCategory=null&defensiveStatisticCategory=INTERCEPTIONS&season=2014&seasonType=REG&tabSeq=2&qualified=false&Submit=Go](http://www.nfl.com/stats/categorystats?archive=false&conference=null&role=OPP&offensiveStatisticCategory=null&defensiveStatisticCategory=INTERCEPTIONS&season=2014&seasonType=REG&tabSeq=2&qualified=false&Submit=Go)

I have attempted this about a million times and can't figure it out.

This is the best i could do:

    
    
    from bs4 import BeautifulSoup
    import urllib2
    import requests
    import pymongo
    import re
    
    soup = BeautifulSoup(urllib2.urlopen('http://www.nfl.com/stats/categorystats?archive=false&conference=null&role=OPP&offensiveStatisticCategory=null&defensiveStatisticCategory=INTERCEPTIONS&season=2014&seasonType=REG&tabSeq=2&qualified=false&Submit=Go').read())
    
    find = soup('a', text="Miami Dolphins")
    
    print find
    

I don't know how to find/call the 10th (9th in python) < td > tag after Miami
Dolphins.

The table code looks something like this.

    
    
    <table id="result" style="width:100%" cellpadding="0" class"data-table1"
    cellspacing="0">
       <caption class="thd1">...</caption>
       <tbody>...</tbody>
       <tbody>
          <tr class="odd">...</tr>
          <tr class="even">...</tr>
          <tr class="odd">...</tr>
          <tr class="even">...</tr>
          <tr class="odd">...</tr>
          <tr class="even">...</tr>
          <tr class="odd">...</tr>
          <tr class="even">...</tr>
          <tr class="odd">...</tr>
          <tr class="even">...</tr>
          <tr class="odd">...</tr>
          <tr class="even">...</tr>
          <tr class="odd">...</tr>
          <tr class="even">...</tr>
          <tr class="odd">...</tr>
          <tr class="even">
             <td>14</td>
             <td>
                <a href="/teams/miamidolphins/profile?team=MIA onclick=
                "s_objectID="http://www.nfl.com/teams.miamidolphins/profile?
                team=MIA_1";return this.s_oc?this.s_oc(e):true">Miami Dolphins</a>    *********I want to grab team name**********
             </td>
             <td>
    
             16
    
             </td>
             <td>24.2</td>
             <td>
    
             388
    
             </td>
             <td class="right">...</td>
             <td class="right">...</td>
             <td class="right">...</td>
             <td class="right">...</td>
             <td class="right">...</td>
             <td class="right">...</td>
             <td class="sorted right">...</td>
               "
    
    
               14  ****I want to grab 10th number/<td> tag after team name****
    
    
                                            "
             </td>
             <td class="right">...</td>
             <td class="right">...</td>
             <td class="right">...</td>
             <td class="right">...</td
          </tr>
    


Answer: Try this

    
    
    import urllib2
    from lxml import etree
    
    url = 'http://www.nfl.com/stats/categorystats?archive=false&conference=null&role=OPP&offensiveStatisticCategory=null&defensiveStatisticCategory=INTERCEPTIONS&season=2014&seasonType=REG&tabSeq=2&qualified=false&Submit=Go'
    response = urllib2.urlopen(url)
    htmlparser = etree.HTMLParser()
    
    tree = etree.parse(response,htmlparser)
    
    text = tree.xpath('//a[contains(text(),"Miami Dolphins")]/parent::td/following-sibling::td[10]/text()')
    if text:
        print text[0].strip()
    

