Python 2.7 decode error using UTF-8 header: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3
Question: Traceback:

    
    
    Traceback (most recent call last):
      File "venues.py", line 22, in <module>
        main()
      File "venues.py", line 19, in main
        print_category(category, 0)
      File "venues.py", line 13, in print_category
        print_category(subcategory, ident+1)
      File "venues.py", line 10, in print_category
        print u'%s: %s' % (category['name'].encode('utf-8'), category['id'])
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128)
    

Code:

    
    
    # -*- coding: utf-8 -*-
    
    # Using https://github.com/marcelcaraciolo/foursquare
    import foursquare 
    
    # Prints categories and subcategories
    def print_category(category, ident):
        for i in range(0,ident):
            print u'\t',
        print u'%s: %s' % (category['name'].encode('utf-8'), category['id'])
    
        for subcategory in category.get('categories', []):
            print_category(subcategory, ident+1)
    
    def main():
        client = foursquare.Foursquare(client_id='id',
                                       client_secret='secret')
        for category in client.venues.categories()['categories']:
            print_category(category, 0)
    
    if __name__ == '__main__':
        main()
    


Answer: The trick is, keep all your string processing in the source completely
Unicode. Decode to Unicode when reading input (files/pipes/console) and encode
when writing output. If `category['name']` is Unicode, keep it that way
(remove `.encode('utf8').

Also Per your comment:

> However, the error still occurs when I try to do: python venues.py >
> categories.txt, but not when output goes to the terminal: python venues.py

Python can usually determine the terminal encoding and will automatically
encode to that encoding, which is why writing to the terminal works. If you
use shell redirection to output to a file, you need to tell Python the I/O
encoding you want via an environment variable, for example:

    
    
    set PYTHONIOENCODING=utf8
    python venues.py > categories.txt
    

Working example, using my US Windows console that uses `cp437` encoding. The
source code is saved in "UTF-8 without BOM". It's worth pointing out that the
_source code_ bytes are UTF-8, but declaring the source encoding and using a
Unicode string in allows Python to decode the source correctly, and encode the
`print` output automatically to the terminal using its default encoding

    
    
    #coding:utf8
    import sys
    print sys.stdout.encoding
    print u'üéâäàåçêëèïîì'
    

Here Python uses the default terminal encoding, but when redirected, does not
know what the encoding is, so defaults to `ascii`:

    
    
    C:\>python example.py
    cp437
    üéâäàåçêëèïîì
    
    C:\>python example.py >out.txt
    Traceback (most recent call last):
      File "example.py", line 4, in <module>
        print u'├╝├⌐├ó├ñ├á├Ñ├º├¬├½├¿├»├«├¼'
    UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-12: ordinal not in range(128)
    
    C:\>type out.txt
    None
    

Since we're using shell redirection, use a shell variable to tell Python what
encoding to use:

    
    
    C:\>set PYTHONIOENCODING=cp437
    
    C:\>python example.py >out.txt
    
    C:\>type out.txt
    cp437
    üéâäàåçêëèïîì
    

We can also force Python to use another encoding, but in this case the
terminal doesn't know how to display `UTF-8`. The terminal is still decoding
the bytes in the file using `cp437`:

    
    
    C:\>set PYTHONIOENCODING=utf8
    
    C:\>python example.py >out.txt
    
    C:\>type out.txt
    utf8
    ├╝├⌐├ó├ñ├á├Ñ├º├¬├½├¿├»├«├¼
    

