title: How to schedule Scrapy crawl execution programmatically
body: <p>I want to create a scheduler script to run the same spider multiple times in a sequence.</p>
  
  <p>So far I got the following:</p>
  
  <pre><code>#!/usr/bin/python3
  """Scheduler for spiders."""
  import time
  
  from scrapy.crawler import CrawlerProcess
  from scrapy.utils.project import get_project_settings
  
  from my_project.spiders.deals import DealsSpider
  
  
  def crawl_job():
      """Job to start spiders."""
      settings = get_project_settings()
      process = CrawlerProcess(settings)
      process.crawl(DealsSpider)
      process.start() # the script will block here until the end of the crawl
  
  
  if __name__ == '__main__':
  
      while True:
          crawl_job()
          time.sleep(30) # wait 30 seconds then crawl again
  </code></pre>
  
  <p>For now the first time the spider executes properly, then after the time delay, the spider starts up again but right before it would start scraping I get the following error message:</p>
  
  <pre><code>Traceback (most recent call last):
    File "scheduler.py", line 27, in &lt;module&gt;
      crawl_job()
    File "scheduler.py", line 17, in crawl_job
      process.start() # the script will block here until the end of the crawl
    File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 285, in start
      reactor.run(installSignalHandlers=False)  # blocking call
    File "/usr/local/lib/python3.5/dist-packages/twisted/internet/base.py", line 1193, in run
      self.startRunning(installSignalHandlers=installSignalHandlers)
    File "/usr/local/lib/python3.5/dist-packages/twisted/internet/base.py", line 1173, in startRunning
      ReactorBase.startRunning(self)
    File "/usr/local/lib/python3.5/dist-packages/twisted/internet/base.py", line 684, in startRunning
      raise error.ReactorNotRestartable()
  twisted.internet.error.ReactorNotRestartable
  </code></pre>
  
  <p>Unfortunately I'm not familiar with the <code>Twisted</code> framework and its <code>Reactor</code>s, so any help would be appreciated!</p>