title: Craw data from urls by passing URL to Scrapy from other *.py file
body: <p>I'm using Scrapy to craw data from website, and this is my code at file spider.py in folder spider of Scrapy</p>
  
  <pre class="lang-py prettyprint-override"><code>class ThumbSpider(scrapy.Spider):
      userInput = readInputData('input/user_input.json')
      name = 'thumb'
      # start_urls = ['https://vietnamnews.vn/politics-laws', 'https://vietnamnews.vn/society']
  
      def __init__(self, *args, **kwargs): 
          super(ThumbSpider, self).__init__(*args, **kwargs)
          self.start_urls = kwargs.get('start_urls')
  
      def start_requests(self):
          for url in self.start_urls:
              yield scrapy.Request(url=url, callback=self.parse)
  
      def parse(self, response):
          for cssThumb in self.userInput['cssThumb']: # browse each cssThumb which user provides
              items = response.css('{0}::attr(href)'.format(cssThumb)).getall() # access it
  
              for item in items:
                  item = response.urljoin(item)
                  yield scrapy.Request(url=item, callback=self.parse_details)
  
      def parse_details(self, response):
          data = response.css('div.vnnews-text-post p span::text').extract()
  
          with open('result/page_content.txt', 'a') as outfile:
              json.dump(data, outfile)
  
          yield data
  </code></pre>
  
  <p>I call class <code>ThumbSpider</code> in file <strong>main.py</strong> and run this file in terminal</p>
  
  <pre class="lang-py prettyprint-override"><code>import json
  import os
  import modules.misc as msc
  from scrapy.crawler import CrawlerProcess
  from week_7.spiders.spider import NaviSpider, ThumbSpider
  
  process2 = CrawlerProcess()
  
  process2.crawl(ThumbSpider, start_urls=['https://vietnamnews.vn/politics-laws', 'https://vietnamnews.vn/society'])
  process2.start()
  </code></pre>
  
  <p>My program doesn't get anything from 2 urls, but when I uncomment <code>start_urls = ['https://vietnamnews.vn/politics-laws', 'https://vietnamnews.vn/society']</code> and delete <code>__init__</code> and <code>start_requests</code> methods in class <code>ThumbSpider</code> and in file <strong>main.py</strong> edit <code>process2.crawl(ThumbSpider, start_urls=msc.getUserChoices())</code> into <code>process2.crawl(ThumbSpider)</code> it worked well. I don't know what happening. Anyone can help me, thank you so much</p>