title: How to extend scrapy feed export to send crawl results in POST requests - open and store not called?
body: <p>I have written a spider and it works fine. I now wish to send the results of that scrape to a rest API via POST request.</p>
  <p>I'm under the impression that extending the feed export functionality to have it make POST requests instead of writing to a file/sending it to S3/etc.</p>
  <p>I am sure that my settings.py and configuration is correct, but for some reason the <code>open</code> and <code>close</code> functions are never called, even when I literally copy and paste and rename preexisting feed export classes from feedexport.py.</p>
  <p>Please see code below:</p>
  <p><code>settings.py</code></p>
  <pre><code>EXTENSIONS = {'project.extensions.ExportToApi.ExportScrape':400}
  FEED_STORAGES = {'http': 'project.extensions.ExportToApi.ExportScrape'}
  FEED_API_URL = 'http://localhost:8000'
  </code></pre>
  <p>(there's a simpleserver at localhost:8000 there printing out all the GET and POST requests it receives)</p>
  <p><code>/project/extensions/ExportToApi.py</code></p>
  <pre><code>from scrapy.extensions.feedexport import BlockingFeedStorage
  
  class ExportScrape(BlockingFeedStorage):
      def __init__(self, crawler, uri):
          self.crawler = crawler
          self.url = uri
  
      @classmethod
      def from_crawler(cls, crawler):
          uri_from_settings = crawler.settings['FEED_API_URL']
          return cls(crawler, uri_from_settings)
  
      def _store_in_thread(self, file):
          file.seek(0)
          import requests
          r = requests.post(self.url, data=file)
          file.close()
  
  </code></pre>
  <p>When added, print statements are run from <code>__init__</code> and <code>from_crawler</code>, but not <code>_store_in_thread</code>, nor <code>open</code> or <code>store</code> if they are implemented.</p>