How to fix my csv output to make it useable?
Question: This might be very easy to fix but I am new to Python, and new to programming.
Using BS4 I have managed to, with help, get this code together. It fetches all
the information I want it to fetch but the output is not very usable.

Here is the code:

    
    
    import requests
    from bs4 import BeautifulSoup
    import StringIO
    import unicodecsv as csv
    
    
    for numb in range(1, 2):
        urls= "http://www.blocket.se/bostad/uthyres?cg_multi=3020&cg_multi=3100&cg_multi=3120&cg_multi=3060&cg_multi=3070&sort=&ss=&se=&ros=&roe=&bs=&be=&mre=&q=&q=&q=&save_search=1&l=0&md=th&o=" +str(numb) +"&f=p&f=c&f=b&ca=11&w=3"
    
        r = requests.get(urls)
        soup=BeautifulSoup(r.text, 'html.parser')
    
        data = soup.find_all("div", {"itemtype": "http://schema.org/Offer"})
    
        data_list = []
        for item in data:
            data_item = {}
            try:
                data_item['category'] = item.contents[3].find_all("span", {"class": "subject-param category"})[0].text
            except:
                data_item['category']= 'NaN'
            try:
                data_item['address'] =  item.contents[3].find_all("span", {"class": "subject-param address separator"})[0].text
            except:
                data_item['address']= 'NaN'
            try:
                data_item['rooms'] = item.contents[3].find_all("span", {"class": "li_detail_params first rooms"})[0].text
            except:
                data_item['rooms']= 'NaN'
            try:
                data_item['monthly_rent'] = item.contents[3].find_all("span", {"class": "li_detail_params monthly_rent"})[0].text
            except:
                data_item['monthly_rent']= 'NaN'
            try:
                data_item['size'] = item.contents[3].find_all("span", {"class": "li_detail_params size"})[0].text  
            except:
                data_item['size']= 'NaN'
            try:
                data_item['weekly_rent_offseason'] = item.contents[3].find_all("span", {"class": "li_detail_params first weekly_rent_offseason"})[0].text
            except:
                data_item['weekly_rent_offseason']= 'NaN'
            try:
                data_item['time'] = item.contents[3].find_all("time", {"class": "jlist_date_image"})[0].text
            except:
                data_item['time']= 'NaN'
            data_list.append(data_item)
        out = StringIO.StringIO()
        csv_writer = csv.writer(out)
        [csv_writer.writerow(data.values()) for data in data_list]
        print out.getvalue()
    

And here is a sample of what it outputs:

    
    
    lägenhet
            ",3 800 kr/mån,Idag 13:58,1 rum,NaN,"
                    Stockholms stad - Älvsjö, Farsta, Vantör
    
                    ",NaN
    "
                radhus
            ",4 700 kr/mån,Idag 13:47,1 rum,NaN,"
                    Stockholms stad - Kista, Hässelby, Vällingby, Spånga
    
                    ",NaN
    "
                villa
            ",NaN,Idag 13:46,8 rum,NaN,"
                    Vellinge
    
                    ",250 m²
    "
                lägenhet
            ",6 000 kr/mån,Idag 13:41,1 rum,NaN,"
                    Stockholms stad - Kista, Hässelby, Vällingby, Spånga
    
                    ",25 m²
    "
                lägenhet
            ",3 500 kr/mån,Idag 13:40,1 rum,NaN,"
                    Örebro
    
                    ",NaN
    

Which corresponds to:

    
    
        category
    ",monthlyrent,time,rooms,weekly_rent_offseason,"
            address
    
            ",size
    

I would like it to read something like (the order is not important):

    
    
    category, address, size, monthly_rent, rooms, time, weekly_rent_offseason
    

Is this possible?

Here is the content of `data_list` :

    
    
    >>> print data_list
    [{'category': u'\n\t\t\tl\xe4genhet\n\t\t', 'monthly_rent': u'11 500 kr/m\xe5n',
    'time': u'Ig\xe5r 21:32', 'rooms': u'2 rum', 'weekly_rent_offseason': 'NaN',
    'address': u'\n\t\t\t\tStockholms stad - H\xe4gersten, Liljeholmen\n\t\t\t\t\n\t\t\t\t',
    'size': u'60 m\xb2'}, {'category': u'\n\t\t\tradhus\n\t\t', 'monthly_rent': 'NaN',
    'time': u'Ig\xe5r 21:32', 'rooms': u'4 rum', 'weekly_rent_offseason': 'NaN',
    'address': u'\n\t\t\t\tBorgholm\n\t\t\t\t\n\t\t\t\t', 'size': u'125 m\xb2'}, ...
    


Answer: You need to strip your strings, and you can use
[`csv.DictWriter()`](https://docs.python.org/2/library/csv.html#csv.DictWriter)
to write dictionaries in a specific order to an output file.

Refactoring your code to use more a more Pythonic approach and all the
Schema.org item properties offered:

    
    
    for numb in range(1):
        params = {
            'cg_multi': [3020, 3100, 3120, 3060, 3070],
            'save_search': 1, 'l': 0, 'md': 'th', 'ca': 11, 'w': 3,
            'f': ['p', 'c', 'b'],
            'o': numb + 1,
        }
        url = "http://www.blocket.se/bostad/uthyres"
        r = requests.get(url, params=params)
        soup = BeautifulSoup(r.content, 'html.parser')
    
        # All your entries live in the itemOffered div
        data = soup.select('div[itemtype=http://schema.org/Offer] div[itemprop=itemOffered]')
    
        out = StringIO.StringIO()
        fields = ('category', 'address', 'size', 'monthly_rent', 'rooms', 'time', 'weekly_rent_offseason')
        csv_writer = csv.DictWriter(out, fieldnames=fields)
    
        for entry in data:
            row = {}
            for field in fields:
                if field == 'time':
                    time_elem = entry.find('time')
                    value = (time_elem or {}).get('datetime', 'NaN')
                else:
                    span_elem = entry.find(class_=field)
                    value = span_elem.text.strip() if span_elem else 'NaN'
                row[field] = value
            csv_writer.writerow(row)
    
        print out.getvalue()
    

Note that the `data` variable already selects the `itemprop="itemOffer"` div
elements, rather than having to navigate to `.contents[3]` each time.

I also opted to extract the `datetime` attribute for the `<time>` elements
rather than use the 'human readable' relative time version in the visible
text; having the actual date and time is more useful when extracting than
_today_.

This produces:

    
    
    lägenhet,Sundsvall,170 m²,NaN,7 rum,2015-04-03 14:43:34,NaN
    radhus,Gotland,66 m²,NaN,2 rum,2015-04-03 14:43:15,NaN
    lägenhet,Gävle,12 m²,3 200 kr/mån,1 rum,2015-04-03 14:34:46,NaN
    lägenhet,Sundbyberg,30 m²,4 500 kr/mån,1 rum,2015-04-03 14:29:07,NaN
    villa,Österåker,9 m²,3 500 kr/mån,1 rum,2015-04-03 14:25:28,NaN
    villa,Söderköping,75 m²,2 700 kr/mån,3 rum,2015-04-03 14:00:25,NaN
    lägenhet,"Stockholms stad - Älvsjö, Farsta, Vantör",NaN,3 800 kr/mån,1 rum,2015-04-03 13:58:22,NaN
    radhus,"Stockholms stad - Kista, Hässelby, Vällingby, Spånga",NaN,4 700 kr/mån,1 rum,2015-04-03 13:47:37,NaN
    villa,Vellinge,250 m²,NaN,8 rum,2015-04-03 13:46:35,NaN
    lägenhet,"Stockholms stad - Kista, Hässelby, Vällingby, Spånga",25 m²,6 000 kr/mån,1 rum,2015-04-03 13:41:42,NaN
    lägenhet,Örebro,NaN,3 500 kr/mån,1 rum,2015-04-03 13:40:02,NaN
    lägenhet,Helsingborg,10 m²,3 000 kr/mån,1 rum,2015-04-03 13:29:44,NaN
    villa,Vingåker,140 m²,7 500 kr/mån,4 rum,2015-04-03 13:27:53,NaN
    villa,Svalöv,200 m²,9 800 kr/mån,5 rum,2015-04-03 13:26:38,NaN
    lägenhet,Sigtuna,49.5 m²,6 000 kr/mån,1 rum,2015-04-03 13:24:49,NaN
    lägenhet,Kristianstad,NaN,NaN,1 rum,2015-04-03 13:14:14,NaN
    villa,Stockholms stad - Bromma,125 m²,25 000 kr/mån,4 rum,2015-04-03 13:13:34,NaN
    lägenhet,Oskarshamn,18 m²,3 000 kr/mån,1 rum,2015-04-03 13:05:50,NaN
    villa,Göteborgs stad - Västra Göteborg,10 m²,NaN,1 rum,2015-04-03 13:02:07,NaN
    gård,Sala,140 m²,NaN,7 rum,2015-04-03 13:00:34,NaN
    lägenhet,"Stockholms stad - Vasastan, Norrmalm",36 m²,14 900 kr/mån,"1,5 rum",2015-04-03 12:57:55,NaN
    lägenhet,Solna,60 m²,12 500 kr/mån,2 rum,2015-04-03 12:48:57,NaN
    lägenhet,Danderyd,50 m²,11 000 kr/mån,1 rum,2015-04-03 12:46:05,NaN
    lägenhet,Luleå,80 m²,6 500 kr/mån,3 rum,2015-04-03 12:45:03,NaN
    villa,Söderhamn,172 m²,NaN,6 rum,2015-04-03 12:42:33,NaN
    lägenhet,"Stockholms stad - Enskede, Årsta, Skarpnäck",51 m²,14 000 kr/mån,2 rum,2015-04-03 12:39:09,NaN
    lägenhet,Umeå,13 m²,2 400 kr/mån,1 rum,2015-04-03 12:37:43,NaN
    lägenhet,Linköping,65 m²,6 300 kr/mån,2 rum,2015-04-03 12:36:02,NaN
    lägenhet,Botkyrka,58.5 m²,NaN,2 rum,2015-04-03 12:34:11,NaN
    lägenhet,Upplands Väsby,80 m²,5 000 kr/mån,3 rum,2015-04-03 12:32:21,NaN
    lägenhet,Strömsund,105 m²,4 000 kr/mån,3 rum,2015-04-03 12:32:21,NaN
    lägenhet,Lund,86 m²,6 500 kr/mån,3 rum,2015-04-03 12:32:18,NaN
    lägenhet,Uppsala,68 m²,6 000 kr/mån,2 rum,2015-04-03 12:28:13,NaN
    lägenhet,"Stockholms stad - Maria, Gamla Stan, Högalid",NaN,25 000 kr/mån,2 rum,2015-04-03 12:24:45,NaN
    lägenhet,Halmstad,21 m²,3 300 kr/mån,1 rum,2015-04-03 12:21:28,NaN
    lägenhet,Uppsala,40 m²,5 000 kr/mån,2 rum,2015-04-03 12:20:40,NaN
    lägenhet,Uppsala,92 m²,NaN,3 rum,2015-04-03 12:17:53,NaN
    lägenhet,Eskilstuna,NaN,NaN,1 rum,2015-04-03 11:48:42,NaN
    lägenhet,Helsingborg,66 m²,7 350 kr/mån,3 rum,2015-04-03 11:44:17,NaN
    villa,Göteborgs stad - Hisingen,110 m²,9 500 kr/mån,4 rum,2015-04-03 11:43:30,NaN
    lägenhet,Göteborgs stad - Västra Göteborg,53 m²,10 000 kr/mån,"2,5 rum",2015-04-03 11:20:36,NaN
    lägenhet,"Stockholms stad - Kista, Hässelby, Vällingby, Spånga",NaN,4 900 kr/mån,1 rum,2015-04-03 11:17:11,NaN
    lägenhet,"Stockholms stad - Hägersten, Liljeholmen",55 m²,12 500 kr/mån,2 rum,2015-04-03 11:16:07,NaN
    lägenhet,Örebro,70 m²,4 000 kr/mån,3 rum,2015-04-03 11:15:02,NaN
    lägenhet,Linköping,"46,5 m²",NaN,1 rum,2015-04-03 11:13:13,NaN
    lägenhet,Kristinehamn,34 m²,3 600 kr/mån,"1,5 rum",2015-04-03 11:09:05,NaN
    lägenhet,Jönköping,NaN,NaN,1 rum,2015-04-03 11:02:07,NaN
    lägenhet,Göteborgs stad - Innerstaden,42 m²,7 000 kr/mån,1 rum,2015-04-03 10:47:58,NaN
    lägenhet,Umeå,70 m²,NaN,4 rum,2015-04-03 10:39:31,NaN
    lägenhet,Göteborgs stad - Västra Centrum,55 m²,7 000 kr/mån,2 rum,2015-04-03 10:16:53,NaN
    

