title: Scrapy managing dynamic spiders
body: <p>I am building a project where I need a web crawler which crawls a list of different webpages. This list can change at any time. How is this best implemented with scrapy? Should I create one spider for all websites or dynamically create spiders?</p>
  
  <p>I have read about <a href="http://scrapyd.readthedocs.org/en/latest/" rel="nofollow">scrapyd</a>, and I guess that dynamically creating spiders is the best approach. I would need a hint about how to implement it though. </p>