title: Excluding specific child node with XPath and Scrapy/lxml
body: <p>I have some Python code that scrapes bbcode forums using scrapy, and I need an Xpath expression that gives me just the text of the posts, excluding the text from quotes. The HTML looks like this:</p>
  
  <pre><code>&lt;td class="postbody"&gt;
     hi this is a response
     &lt;div class="bbc-block"&gt;
        &lt;blockquote&gt;
           blah blah blah here's a quote
           &lt;br&gt;
        &lt;/blockquote&gt;
     &lt;/div&gt;
     &lt;br&gt;
     and now I'm responding to what I quoted
  &lt;/td&gt;
  &lt;td class="postbody"&gt;
     &lt;div class="bbc-block"&gt;
        &lt;blockquote&gt;
           and now I'm responding to what I quoted
           &lt;br&gt;
        &lt;/blockquote&gt;
     &lt;/div&gt;
     &lt;br&gt;
     wow what a great response
  &lt;/td&gt;
  </code></pre>
  
  <p>This occurs many times per page, for each post. What I ultimately want is just the text for each of these td nodes with the blockquote excluded:</p>
  
  <ol>
  <li>hi this is a response \n and now I'm responding to what I quoted</li>
  <li>wow what a great response</li>
  </ol>
  
  <p>The Python code I have to extract these blocks is as follows--first I converted it from scrapy's HtmlResponse to lxml's HtmlElement class, because that was the only way I could figure out to use the lxml.html.text_content() method:</p>
  
  
  
  <pre><code>import lxml.html as ht
  
  def posts_from_response(self, response):
      dom = ht.fromstring(response.body)
      posts = dom.xpath('//td[@class="postbody"]')
      posts_text = [p.text_content() for p in posts]
      return posts_text
  </code></pre>
  
  <p>I've searched for solutions extensively for a few days, and tried about a dozen variations of </p>
  
  <pre><code>'//td[@class="postbody"][not(@class="bbc-block")]'
  </code></pre>
  
  <p>appended to that in various ways, but nothing gets me exactly what I want with the grouping that I want.</p>
  
  <p>Is there either 1. a way to get this with a single statement, or 2. a way to execute a second Xpath selector on my <code>posts</code> list to exclude the bbc-block nodes?</p>