python lxml.html: returns null list for yahoo finance
Question: The following code returns an empty list;

    
    
    import lxml.html
    url = 'http://finance.yahoo.com/q/pr?s=AYR+Profile'
    content = lxml.html.parse(url)
    sector = content.xpath('//*[@id="yfncsumtab"]/tbody/tr[2]/td[1]/table[2]/tbody/tr/td/table/tbody/tr[2]/td[2]/a')
    print sector
    []
    

I cross checked on the console in chrome ..

    
    
    $x('//*[@id="yfncsumtab"]/tbody/tr[2]/td[1]/table[2]/tbody/tr/td/table/tbody/tr[2]/td[2]/a')[0].text
    

and I get the output as;

    
    
    'Services'
    

Where am I messing up?


Answer: In the `html` code resolved by `lxml` there aren't `<tbody>` elements, so you
have to omit them in the `xpath` expression. Also, add `text()` at the end to
extract the content of the link:

    
    
    sector = content.xpath('//*[@id="yfncsumtab"]/tr[2]/td[1]/table[2]/tr/td/table/tr[2]/td[2]/a/text()')
    

