title: pulling multiple values from python ElementTree with lxml and xpath
body: <p>I am almost certainly doing this horribly wrong, and the cause of my problem is my own ignorance, but reading python docs and examples isn't helping.</p>
  
  <p>I am web-scraping.  The pages I am scraping have the following salient elements:</p>
  
  <pre><code>&lt;div class='parent'&gt;
     &lt;span class='title'&gt;
        &lt;a&gt;THIS IS THE TITLE&lt;/a&gt;
     &lt;/span&gt;
     &lt;div class='copy'&gt;
        &lt;p&gt;THIS IS THE COPY&lt;/p&gt;
     &lt;/div&gt;
  &lt;/div&gt;
  </code></pre>
  
  <p>My objective is to pull the text nodes from 'title' and 'copy', grouped by their parent div.  In the above example, I should like to retrieve a tuple <code>('THIS IS THE TITLE', 'THIS IS THE COPY')</code></p>
  
  <p>Below is my code</p>
  
  <pre><code>## 'tree' is the ElementTree of the document I've just pulled 
  xpath = "//div[@class='parent']"
  filtered_html = tree.xpath(xpath)
  
  arr = []
  
  for i in filtered_html:
  
     title_filter = "//span[@class='author']/a/text()"  # xpath for title text
     copy_filter = "//div[@class='copy']/p/text()"      # xpath for copy text
  
     title = i.getroottree().xpath(title_filter)
     copy = i.getroottree().xpath(copy_filter)
     arr.append((title, copy))
  </code></pre>
  
  <p>I'm expecting <code>filtered_html</code> to be a list of <em>n</em> elements (which it is).  I'm then trying to iterate over that list of elements and for each one, convert it to an ElementTree and retrieve the title and copy text with another xpath expression.  So at each iteration, I'm expecting <code>title</code> to be a list of length 1, containing the title text for element <em>i</em>, and <code>copy</code> to be a corresponding list for the copy text.</p>
  
  <p>What I end up with: at every iteration, <code>title</code> is a list of length <em>n</em> containing all elements in the document matching the <code>title_filter</code> xpath expression, and <code>copy</code> is a corresponding list of length <em>n</em> for the copy text.</p>
  
  <p>I'm sure that by now, anyone who knows what they're doing with xpath and etree can recognise I'm doing something horrible and mistaken and stupid.  If so, can they please tell me how I should be doing this instead?</p>