title: HTML parsing with lxml, python, .tail being broken up by <br> tags
body: <p>I have a website that I am trying to scrape (while not really understanding html) but I have done a ton of reading and made some progress. It's a messy site but the important part looks like this:  </p>
  
  <pre><code>&lt;h1&gt;
      &lt;b&gt;DESCRIPTOR1: &lt;/b&gt; 
         " important content "
      &lt;br&gt;
      &lt;b&gt;DESCRIPTOr2: &lt;/b&gt; 
         " important content"
      &lt;hr&gt;
  &lt;/h1&gt;
  
  &lt;b&gt;Title1&lt;/b&gt;
      " A lot of important text"
  &lt;br&gt;
  &lt;br&gt;
  &lt;b&gt;Title2&lt;/b&gt;
      "A lot of important text"
  &lt;br&gt;
  &lt;br&gt;
  &lt;b&gt;Title3&lt;/b&gt;
  &lt;br&gt;
      "1. List of text pertaining to Title3 "
  &lt;br&gt;
      "2. List of items for Title 3"
  &lt;br&gt; 
      "3. the number of listed items is variable for every page"
  &lt;br&gt;
      "4. Sometimes no list at all"
  &lt;br&gt;
  &lt;br&gt;
  &lt;b&gt; Next Title: &lt;/b&gt;
  
  ....and so on
  </code></pre>
  
  <p>Now I can get pretty close to what I want my final result to be except for when I get to Title 3 and there is a <code>&lt;br&gt;</code> before the content that goes with Title 3. This is how I am approaching it:</p>
  
  <pre><code>import lxml.html
  
  htmltree = lxml.html.parse('sample.html')
  
  items = htmltree.xpath('//*[@id="sampletext"]/b')
  
  for node in items:
  
      print (node.text.strip())
      print node.tail
  </code></pre>
  
  <p>Now my 2 problems are (1): I can't strip out whitespaces from the <code>.tail's</code> and (2): I get "None" returned for Title3 because there is no <code>.tail</code> before the next element which is a <code>&lt;br&gt;</code>. Ideally, I would be able to add up any text NOT in between element tags until I get to my next Identifier-Tag which in this case is <code>&lt;b&gt;</code>. Hope that makes sense. Any pointers please?</p>