Trouble reading in Unicode strings from CSV file to DictReader in Python
Question: I have a CSV file I'm trying to read in using DictReader.

But doing just this:

    
    
    with("BeerRatings.csv", "r", "utf-8") as f:
        reader = csv.DictReader(f)
        for line in reader:
            print line
    

gives me some ugly unicode as such:

    
    
    {'Rating': '4', 'Brewery': 'Tr\xc3\xb6egs Brewing Company', 'Beer name': 'Tr\xc3\xb6egs Hopback Amber Ale'}
    {'Rating': '4.59', 'Brewery': 'Brasserie Dieu Du Ciel', 'Beer name': 'P\xc3\xa9ch\xc3\xa9 Mortel - Bourbon Barrel Aged'} etc.
    

So, reading on stackoverflow, I editted my code to this, using the codecs
module:

    
    
    import codecs
    
    with codecs.open("BeerRatings.csv", "r", "utf-8") as f:
        reader = csv.DictReader(f)
        for line in reader:
            print line
    

But this is giving me a `UnicodeEncodeError: 'ascii' codec can't encode
character u'\xea' in position 9: ordinal not in range(128)`.

Any tips on how to go fix this?

UPDATE aka more flailing around:

    
    
    def UnicodeDictReader(utf8_data, **kwargs):
        csv_reader = csv.DictReader(utf8_data, **kwargs)
        for row in csv_reader:
            yield {key: unicode(value, 'utf-8') for key, value in row.iteritems()}
    
    with open("BeerRatings.csv", "r") as f:
        reader = UnicodeDictReader(f)
        for line in reader:
            print line
    

THis still gives me a less than ideal output...

    
    
    {'Rating': u'4', 'Brewery': u'Tr\xf6egs Brewing Company', 'Beer name': u'Tr\xf6egs Hopback Amber Ale'}
    {'Rating': u'4.59', 'Brewery': u'Brasserie Dieu Du Ciel', 'Beer name': u'P\xe9ch\xe9 Mortel - Bourbon Barrel Aged'}
    


Answer: The `csv` module in Python 2.X expects the input file to be opened in binary,
and does not support encodings. It is, however, compatible with UTF-8, but you
have to decode to Unicode yourself:

    
    
    import csv
    
    with open('BeerRatings.csv','rb') as f:
        reader = csv.DictReader(f)
        for line in reader:
            for k,v in line.iteritems():
                print k.decode('utf8'),':',v.decode('utf8')
            print
    

Output:

    
    
    Rating : 4
    Brewery : Tröegs Brewing Company
    Beer name : Tröegs Hopback Amber Ale
    
    Rating : 4.59
    Brewery : Brasserie Dieu Du Ciel
    Beer name : Péché Mortel - Bourbon Barrel Aged
    

### Edit

Per your `UnicodeDictReader`, you still need to print the key/value pairs as I
did or you get the default printing for a `dict`, which shows escaped data via
the `repr()` of the string. Also open in binary mode. It matters on some OSes,
particularly Windows.

    
    
    import csv
    
    def UnicodeDictReader(utf8_data, **kwargs):
        csv_reader = csv.DictReader(utf8_data, **kwargs)
        for row in csv_reader:
            yield {key.decode('utf8'):value.decode('utf8') for key, value in row.iteritems()}
    
    def prettydict(D):
        return u'{' + u', '.join(u"'{}': '{}'".format(k,v) for k,v in D.iteritems()) + u'}'
    
    with open("BeerRatings.csv", "rb") as f:
        reader = UnicodeDictReader(f)
        for line in reader:
            print prettydict(line)
    

Output:

    
    
    {'Rating': '4', 'Brewery': 'Tröegs Brewing Company', 'Beer name': 'Tröegs Hopback Amber Ale'}
    {'Rating': '4.59', 'Brewery': 'Brasserie Dieu Du Ciel', 'Beer name': 'Péché Mortel - Bourbon Barrel Aged'}
    

