title: How to scrape all the content of each link with scrapy?
body: <p>I am new with scrapy I would like to extract all the content of each advertise from this <a href="https://sfbay.craigslist.org/search/jjj?employment_type=2" rel="nofollow noreferrer">website</a>. So I tried the following:</p>
  
  <pre><code>from scrapy.spiders import Spider
  from craigslist_sample.items import CraigslistSampleItem
  
  from scrapy.selector import Selector
  class MySpider(Spider):
      name = "craig"
      allowed_domains = ["craigslist.org"]
      start_urls = ["http://sfbay.craigslist.org/search/npo"]
  
      def parse(self, response):
          links = response.selector.xpath(".//*[@id='sortable-results']//ul//li//p")
          for link in links:
              content = link.xpath(".//*[@id='titletextonly']").extract()
              title = link.xpath("a/@href").extract()
              print(title,content)
  </code></pre>
  
  <p>items:</p>
  
  <pre><code># Define here the models for your scraped items
  
  from scrapy.item import Item, Field
  
  class CraigslistSampleItem(Item):
      title = Field()
      link = Field()
  </code></pre>
  
  <p>However, when I run the crawler I got nothing:</p>
  
  <pre><code>$ scrapy crawl --nolog craig
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  []
  </code></pre>
  
  <p>Thus, my question is: How can I walk over each url, get inside each link and crawl the content and the title?, and which is the best way of do this?.</p>