title: Using Beautiful Soup for HTML tables that lack </td> tags
body: <p>I'm struggling with parsing some flaky HTML tables down to lists with Beautiful Soup.  The tables in question lack a &lt;/td&gt; tag.</p>
  
  <p>Using the following code (not the real tables I'm parsing, but functionally similar):</p>
  
  <pre><code>import bs4
  test = "&lt;table&gt; &lt;tr&gt;&lt;td&gt;1&lt;td&gt;2&lt;td&gt;3&lt;/tr&gt; &lt;tr&gt;&lt;td&gt;1&lt;td&gt;2&lt;td&gt;3&lt;/tr&gt; &lt;/table&gt;"
  def walk_table2(text):
      "Take an HTML table and spit out a list of lists (of entries in a row)."
      soup = bs4.BeautifulSoup(text)
      return [[x for x in row.findAll('td')] for row in soup.findAll('tr')]
  
  print walk_table2(test)
  </code></pre>
  
  <p>Gives me:</p>
  
  <pre><code>[[&lt;td&gt;1&lt;td&gt;2&lt;td&gt;3&lt;/td&gt;&lt;/td&gt;&lt;/td&gt;, &lt;td&gt;2&lt;td&gt;3&lt;/td&gt;&lt;/td&gt;, &lt;td&gt;3&lt;/td&gt;], [&lt;td&gt;4&lt;td&gt;5&lt;td&gt;6&lt;/td&gt;&lt;/td&gt;&lt;/td&gt;, &lt;td&gt;5&lt;td&gt;6&lt;/td&gt;&lt;/td&gt;, &lt;td&gt;6&lt;/td&gt;]]
  </code></pre>
  
  <p>Rather than the expected:</p>
  
  <pre><code>[[&lt;td&gt;1&lt;/td&gt;, &lt;td&gt;2&lt;/td&gt;, &lt;td&gt;3&lt;/td&gt;], [&lt;td&gt;1&lt;/td&gt;, &lt;td&gt;2&lt;/td&gt;, &lt;td&gt;3&lt;/td&gt;]]
  </code></pre>
  
  <p>It seems that the lxml parser that Beautiful Soup is using decides to add the &lt;/td&gt; tag before the next instance of &lt;/tr&gt; rather than the next instance of &lt;td&gt;.</p>
  
  <p>At this point, I'm wondering if there a good option to make the parser place the ending td tags in the correct location, or if it would be easier to use a regular expression to place them manually before tossing the string into BeautifulSoup...  Any thoughts?  Thanks in advance!</p>