title: Python/lxml/xpath for parsing Yahoo Finance
body: <p>EDIT: I have provided the EXACT source code I'm using to try to figure out this issue.</p>
  
  <p>I'm trying to extract the data on "total assets" from Yahoo Finance using Python 2.7 and lxml.  An example of a page I'm trying to extract this information from is <strong>http://finance.yahoo.com/q/bs?s=FAST+Balance+Sheet&amp;annual</strong> .</p>
  
  <p>I've already successfully extracted the data on "total assets" from Smartmoney.  An example of a Smartmoney page I'm able to parse is <strong>http://www.smartmoney.com/quote/FAST/?story=financials&amp;timewindow=1&amp;opt=YB&amp;isFinprint=1&amp;framework.view=smi_emptyView</strong> .</p>
  
  <p>Here is a special test script I set up to work on this issue:</p>
  
  <pre><code>    import urllib
      import lxml
      import lxml.html 
  
      url_local1 = "http://www.smartmoney.com/quote/FAST/?story=financials&amp;timewindow=1&amp;opt=YB&amp;isFinprint=1&amp;framework.view=smi_emptyView" 
      result1 = urllib.urlopen(url_local1)
      element_html1 = result1.read()
      doc1 = lxml.html.document_fromstring (element_html1)
      list_row1 = doc1.xpath(u'.//th[div[text()="Total Assets"]]/following-sibling::td/text()')
      print list_row1
  
      url_local2 = "http://finance.yahoo.com/q/bs?s=FAST" 
      result2 = urllib.urlopen(url_local2)
      element_html2 = result2.read()
      doc2 = lxml.html.document_fromstring (element_html2)
      list_row2 = doc2.xpath(u'.//td[strong[text()="Total Assets"]]/following-sibling::td/strong/text()')
      print list_row2
  </code></pre>
  
  <p>I'm able to get the row of data on total assets from the Smartmoney page, but I get just an empty list when I try to parse the Yahoo Finance page.</p>
  
  <p>The source code of the table row on the Smartmoney page is:</p>
  
  <pre><code>    &lt;tr class="odd bold"&gt;
  &lt;th&gt;&lt;div style='font-weight:bold'&gt;Total Assets&lt;/div&gt;&lt;/th&gt;
  &lt;td&gt;  1,684,948&lt;/td&gt;
  &lt;td&gt;  1,468,283&lt;/td&gt;                                
  &lt;td&gt;  1,327,358&lt;/td&gt;                                
  &lt;td&gt;  1,304,149&lt;/td&gt;                                    
  &lt;td&gt;  1,163,061&lt;/td&gt;
      &lt;/tr&gt;
  </code></pre>
  
  <p>The source code of the table row on the Yahoo page is:</p>
  
  <pre><code>    &lt;tr&gt;
  &lt;td colspan="2"&gt;&lt;strong&gt;Total Assets&lt;/strong&gt;&lt;/td&gt;
  &lt;td align="right"&gt;&lt;strong&gt;1,684,948&amp;nbsp;&amp;nbsp;&lt;/strong&gt;&lt;/td&gt;
  &lt;td align="right"&gt;&lt;strong&gt;1,468,283&amp;nbsp;&amp;nbsp;&lt;/strong&gt;&lt;/td&gt;
  &lt;td align="right"&gt;&lt;strong&gt;1,327,358&amp;nbsp;&amp;nbsp;&lt;/strong&gt;&lt;/td&gt;
      &lt;/tr&gt;
  </code></pre>