Writing to a csv file in a customized way using scrapy I've written a script in scrapy to grab different names and links from different pages of a website and write those parsed items in a csv file. When I run my script, I get the results accordingly and find a data filled in csv file. I'm using python 3.5, so when I use scrapy's built-in command to write data in a csv file, I do get a csv file with blank lines in every alternate row. Eventually, I tried the below way to achieve the flawless output (with no blank lines in between). Now, It produces a csv file fixing blank line issues. I hope I did it in the right way. However, if there is anything I can/should do to make it more robust, I'm happy to cope with.

This is my script which provides me with a flawless output in a csv file:

import scrapy ,csv
from scrapy.crawler import CrawlerProcess

class GetInfoSpider(scrapy.Spider):
    name = "infrarail"

    start_urls= ['http://www.infrarail.com/2018/exhibitor-profile/?e={}'.format(page) for page in range(65,70)]

    def __init__(self):
        self.infile = open("output.csv","w",newline="")

    def parse(self, response):
        for q in response.css("article.contentslim"):
            name = q.css("h1::text").extract_first()
            link = q.css("p a::attr(href)").extract_first()
            yield {'Name':name,'Link':link}

            writer = csv.writer(self.infile)
            writer.writerow([name,link])

c = CrawlerProcess({
    'USER_AGENT': 'Mozilla/5.0',   
})
c.crawl(GetInfoSpider)
c.start()


Btw, I used .CrawlerProcess() to be able to run my spider from sublime text editor.
I'd like to mention, that there is a special way of making output files in scrapy - item pipelines. So, in order to make it right, you should write your own pipeline (or modify standard one via subclassing).

Also, you does not close the file, once you're done and you keep it open most of the time. The both problems are handled nicely with pipelines. 

UPD: Well, you've asked for a better way, there it is. Although, if it's not acceptable for some hard-to-explain reasons (it's understandable), here's the other approach, how to make it better. 


Don't leave the file open. There is a method (__del__()) which destroys the spider object. Add the code to close the file before it. 
Another one is to store only the filename in the variable and open / close the file each time you write into it. 
Another option is to use NoSQL database, which does not need to be opened / closed. And after scraping is done - get the output file from it. 
If you have a few values to scrape you can store it in object variable, and then export it before __del__() method. 


All the ways above are NOT welcomed by the actual developer community and may lead to serious problem in future. Use them carefully. Sometimes it's easier (in the long run) to read and understand how it really should be done. 

Maybe it's the exact case?