title: unable to scrape a text file through a given link and store it in an output file
body: <p>I'm trying to collect the weather data for year 2000 from <a href="http://www1.ncdc.noaa.gov/pub/data/uscrn/products/daily01" rel="nofollow noreferrer">this</a> site.</p>
  
  <p>My code for the spider is:</p>
  
  <pre><code>    import scrapy
      from scrapy.contrib.spiders import CrawlSpider, Rule
      from scrapy.contrib.linkextractors import LinkExtractor
  
      class weather(CrawlSpider): 
          name = 'data'
          start_urls = [
         'https://www1.ncdc.noaa.gov/pub/data/uscrn/products/daily01/'
          ]
          custom_settings = {
                 'DEPTH_LIMIT': '2',
          }
          rules = (
          Rule(LinkExtractor(restrict_xpaths=
          ('//table/tr[2]/td[1]/a',)),callback='parse_item',follow=True),
          )
          def parse_item(self, response):
            for b in response.xpath('//table')
  
              yield scrapy.request('/tr[4]/td[2]/a/@href').extract()
              yield scrapy.request('/tr[5]/td[2]/a/@href').extract()
  </code></pre>
  
  <p>The paths with 'yield' are the links to two text file and i want to scrape the data from these text files and store it separately in two different files but I don't know how to continue.  </p>