title: CPU-intensive parsing with scrapy
body: <p>The CONCURRENT_ITEMS section at <a href="http://doc.scrapy.org/en/latest/topics/settings.html#concurrent-items" rel="nofollow">http://doc.scrapy.org/en/latest/topics/settings.html#concurrent-items</a> defines it as:</p>
  
  <blockquote>
    <p>Maximum number of concurrent items (per response) to process in
    parallel in the Item Processor (also known as the Item Pipeline).</p>
  </blockquote>
  
  <p>This confuses me. Does this imply that the items sent to the pipeline are processed in parallel ie. really multiprocessed?</p>
  
  <p>Suppose my parsing involves a lot of lxml querying and xpath'ing. Should I do them in the spider's parse method itself, or should I send an Item with the whole response in it and let custom pipeline classes populate the Item's fields by parsing the response body?</p>