title: How to save python scrapy crawlspider outputs to separate txt files?
body: <p>I have a list of websites in a csv file as below,</p>
  <pre><code>id, url
  100, example1.com
  200, example2.com
  300, example3.com
  ...
  </code></pre>
  <p>I am tring to write a python scrapy crawlspider to download all text in the websites. I need to save each website's text as a separate txt file with id as the file name, such as 100.txt and 200.txt, for further text analysis. Here is my scrapy code,</p>
  <pre><code>import scrapy
  from scrapy.linkextractors import LinkExtractor
  from scrapy.spiders import CrawlSpider, Rule
  import pandas as pd
  
  df = pd.read_csv('Test2.csv')
  df['id'] = df['id'].apply(str)
  
  class Hosp2Spider(CrawlSpider):
      name = 'hosp2'
  
      def __init__(self, url=None, *args, **kwargs):
          for index, row in df.iterrows():
              url = row['url']
              super(Hosp2Spider, self).__init__(*args, **kwargs)
              self.allowed_domains = [url]
              self.start_urls = [&quot;http://&quot; + url]
  
      rules = (
          Rule(LinkExtractor(), callback='parse_item', follow=False),
      )
  
  
      def parse_item(self, response):
          raw = response.xpath('//body//text()').extract()
          out = ','.join(raw)
  
          for index, row in df.iterrows():
              with open(row['id'] + '.txt', 'a+', encoding='utf-8') as f_out:
                  out = f_out.write(out)
  </code></pre>
  <p>Note: I use &quot;follow=False&quot; for test purposes. After debugging, I will update it to follow=True for the whole site.</p>
  <p>I got error messages: &quot;out = f_out.write(out) TypeError: write() argument must be str, not int&quot;. Only first two txt files were generated, but the first one (100.txt) contains the second one's (example2.com) text. The second txt file (200.txt) is empty. How to fix it? Any suggestion is highly appreciated. Thank you.</p>
  <p>#################################
  update on 4/1/2022</p>
  <p>I updated my codes and solved the problem of TypeError: write() argument must be str, not int&quot; by revising to the following code.</p>
  <pre><code>     for index, row in df.iterrows():
           if isinstance(out, str):
               with open(row['id'] + '.txt', 'a+', encoding='utf-8') as f_out:
               out = f_out.write(out)
           else:
               pass
  </code></pre>
  <p>Now, 100.txt, 200.txt, and 300.txt all show up. However, all meaningful outs are in 100.txt.</p>
  <p>200.txt and 300.txt only contain some numbers, such as &quot;016148016148016148016148016148018916289162891628916289162891628916289...&quot;. How can I get the extracted text saved into the corresponding .txt files? Thank you.</p>