title: Scrapy/Python/XPath - How to extract data from within data?
body: <p>I'm new to Scrapy, and I've just started looking into XPath.</p>
  
  <p>I'm trying to extract titles and links from html list items within a div. The following code is how I thought I'd go about doing it, (selecting the ul div, by id, then looping through the list items):</p>
  
  <pre><code>def parse(self, response):
      for t in response.xpath('//*[@id="categories"]/ul'):
          for x in t.xpath('//li'):
              item = TgmItem()
              item['title'] = x.xpath('a/text()').extract()
              item['link'] = x.xpath('a/@href').extract()
              yield item
  </code></pre>
  
  <p>But I received the same results as this attempt:</p>
  
  <pre><code>def parse(self, response):
      for x in response.xpath('//li'):
          item = TgmItem()
          item['title'] = x.xpath('a/text()').extract()
          item['link'] = x.xpath('a/@href').extract()
          yield item
  </code></pre>
  
  <p>Where the exported csv file contains li data from source code top to bottom...</p>
  
  <p>I'm not an expert and I've made a number of attempts, if anyone could shed some light on this it would be appreciated.</p>