title: how can i use scrapy to finish a distributed scraper with celery?
body: <p>Now i want to complete a distributed scraper with scrapy and celery,my current idea is to use master-slave method,can someone tell me is that a good idea?is there a good open-source project for this?</p>