title: Python: Running a scrapy spider indefinitely to monitor page changes
body: <p>I'm using scrapy to check the price and availability of certain clothing products I'm interested in.</p>
  
  <p>The program itself works as intended, but now I'm not sure how to get it to continually loop to work as a page monitor.  I plan on renting out a server to have it run there indefinitely and if there's a change in the availability or price, it will notify me via slack.  I use only one spider, so my code looks something like this:</p>
  
  <pre><code>class MonitorSpider(SitemapSpider):
      name = 'page_monitor'
      sitemap_urls = [
      'https://www.example.com/sitemap.xml'
      'https://www.example.com/sitemap.xml'
      'https://www.example.com/sitemap.xml'
      ]
  
      # Using the sitemap rules so time won't be wasted following links I don't care about
      sitemap_rules =[('keyword', 'parse')]
  
      def parse(self, response):
          magically get price and availability
  </code></pre>
  
  <p>Also, if anyone could settle a debate I had with a friend earlier that'd be pretty helpful too.  He thinks that beautiful soup would run faster for a project like this, but I think scrapy is the better option because the problem I've run into with scrapy in the past is that it's usually TOO fast and can get you temporarily IP banned from the site (I plan on incorporating proxies to get around this).  I know I could make a script that replicates this one using beautiful soup and just test the two, but I'd rather not do so if it's not necessary.</p>
  
  <p>Speed is very important concerning this program. I want to be notified if it's available ASAP.</p>