title: Python Web Scraping - Navigating to next page link and obtaining data
body: <p>I am trying to navigate to links and extracting data (the data is a href download link),this data should be added to a new field besides the previous fields of the first page (from where i got the links),but i am struggling how to do that</p>
  
  <p>Firstable,i've created a parse and extracted all the links of the first page and added it to a field named "Links",this links are redirecting to a page that contains a download Button,so i need the real link of the download button,so what i did here is to create a for loop with the previous links and executing the function yield response.follow but it didn't go well. </p>
  
  <pre><code>import scrapy
  class thirdallo(scrapy.Spider):
      name = "thirdallo"
      start_urls = [
          'https://www.alloschool.com/course/alriadhiat-alaol-ibtdaii',
  
  ]
  
  def parse(self, response):
  
          yield {
            'path': response.css('ol.breadcrumb li a::text').extract(),
            'links': response.css('#top .default .er').xpath('@href').extract() 
  
          }
  
          hrefs=response.css('#top .default .er').xpath('@href').extract()
          for i in hrefs:
              yield response.follow(i, callback=self.parse,meta={'finalLink' :response.css('a.btn.btn-primary').xpath('@href)').extract() })
  </code></pre>