title: What are the best practices for calling an external api?
body: <p>So let's say I want to write a spider that using the Facebook API to calculate the likes on every page of a website. If I import the requests library, I'm able to call the Facebook graph API as follows. </p>
  
  <pre><code>import scrapy
  import json
  import requests
  
  API_KEY="KEY_GOES_HERE"
  
  class WebSite(scrapy.Spider):
      name = "website_page"
      allowed_domains = ["website.com"]
      start_urls = ['https://website.com/']
      def get_likes(self,url):
        base='https://graph.facebook.com/{}?access_token={}'.format(url,API_KEY)
        data=requests.get(base)
        return self.parse_likes(data)
      def parse_likes(self, data):
        data = json.loads(data.text)
        return data['id'],data['share']['comment_count'],data['share']['share_count']
  
      def parse(self, response):
          item= {}
          item['url'] = response.url
          links = response.css('a::attr(href)').extract()
          item['fb_url'],item['shares'],item['comments'] = self.get_likes(response.url)
          for link in links: 
            link = response.urljoin(link)
            item['link'] = link
            yield scrapy.Request(link, callback=self.parse)
          yield item
  </code></pre>
  
  <p>However, I can't seem to get this code to work if, rather than using the requests, I use the scrapy.Request call. Something like this. </p>
  
  <pre><code>import scrapy
  import json
  import requests
  
  API_KEY="KEY_GOES_HERE"
  
  class WebSite(scrapy.Spider):
      name = "website_page"
      allowed_domains = ["website.com"]
      start_urls = ['https://website.com/']
      def get_likes(self,url):
        base='https://graph.facebook.com/{}?access_token={}'.format(url,API_KEY)
        return scrapy.Request(base,callback=self.parse_likes)
      def parse_likes(self, data):
        data = json.loads(data.text)
        return data['id'],data['share']['comment_count'],data['share']['share_count']
  
      def parse(self, response):
          item= {}
          links = response.css('a::attr(href)').extract()
          item['url'] = response.url
          item['fb_data']=self.get_likes(response.url).body
          for link in links: 
            link = response.urljoin(link)
            item['link'] = link
            yield scrapy.Request(link, callback=self.parse)
          yield item
  </code></pre>
  
  <p>In this case, I just get a blank response for the Facebook data. I think i'm missing some understanding about how the scrapy.Request method works relative to the standard requests library. Any ideas? </p>