title: How to use Scrapy Request and get response at same place?
body: <p>I am writing the scrapy crawler to scrape the data from the e-commerce website.
  The website has the color variant and each variant has own price, sizes and stock for that sizes.
  To get the price, sizes, and the stocks for variant need to visit the link of the variant(color).
  And all data needes in one record.
  I have tried using requests but it is slow and sometimes fails to load the page.</p>
  
  <p>I have written the crawler using <code>requests.get()</code> and use the response in the <code>scrapy.selector.Selector()</code> and parsing data.</p>
  
  <p>What my question is, is there any way to use scrapy.Request() to get the response where I use it not at the callback function. I need the response at the same place as below(something like below),</p>
  
  <pre><code>response = scrapy.Request(url=variantUrl)
  sizes = response.xpath('sizesXpath').extract()
  </code></pre>
  
  <p>I know <code>scrapy.Request()</code> require parameter called <code>callback=self.callbackparsefunction</code>
  that will be called when scrapy generates the response to handle that generated response. I do not want to use callback functions I want to handle the response in the current function.</p>
  
  <p>Or is there any way to return the response from the callback function to function where <code>scrapy.Request()</code> is written as below(something like below),</p>
  
  <pre><code>def parse(self, response):
      variants = response.xpath('variantXpath').extract()
      for variant in variants:
          res = scrapy.Request(url=variant,callback=self.parse_color)
          # use of the res response
  
  def parse_color(self, response):
      return response
  </code></pre>