JSON in Python: encoding issue on OS X, no issue on Windows
Question: I have a file `log.json` that contains one line:

    
    
    {"k":"caf\u00e9"}
    

I run the following code on Windows 7 SP1 x64 Ultimate:

    
    
    import json 
    a = json.load(open('log.json', 'r'))
    f = open('test.txt', 'w')
    f.write(a['k'])
    

I don't have any issue.

When I run the same code on Max OS X 10.10 x64:

    
    
    Traceback (most recent call last):
      File "/Users/francky/Documents/workspace/test.py", line 4, in <module>
        f.write(a['k'])
    UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 3: ordinal not in range(128)
    

How comes it works fine on Windows but not on OS X?

* * *

The versions of the Python interpreter as well as of the JSON Python package
are the same on the two OS:

    
    
    import json 
    import sys
    print json.__version__
    print(sys.version)
    

returns on OS X:

    
    
    2.0.9
    2.7.6 (default, Sep  9 2014, 15:04:36) 
    [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]
    

and on Windows:

    
    
    2.0.9
    2.7.6 (default, Nov 10 2013, 19:24:24) [MSC v.1500 64 bit (AMD64)]
    

* * *

The culprit was PyDev, which [used to use the Eclipse workspace or project
setting "Text file encoding" and sets this as the Python default
encoding](https://sw-brainwy.rhcloud.com/tracker/PyDev/315) (fixed in PyDev
3.4.0 and after).

and to support some non-ASCII characted I had switched Python file to UTF-8:

![enter image description here](http://i.stack.imgur.com/qHKmw.png)

which caused Python's `sys.getdefaultencoding()` to be `UTF-8`:

![enter image description here](http://i.stack.imgur.com/KHTNk.png)

FYI: [Dangers of
sys.setdefaultencoding('utf-8')](http://stackoverflow.com/q/28657010/395857)


Answer: I get your OSX failure on Windows, and it _should_ fail because writing a
Unicode string to a file requires an encoding. When you write Unicode strings
to a file Python 2 will implicitly convert it to a byte string using the
default `ascii` codec and fails for non-ASCII characters. Are you sure you are
running Python 2.7? Python 3 gives no error. `io.open` is the Python 3
equivalent of `open` on Python 2 and defaults to using
`sys.getfilesystemencoding()`. Here's how to fix Python 2:

    
    
    import json
    import io
    data = r'{"k":"caf\u00e9"}'
    a = json.loads(data)
    with io.open('test.txt','w') as f:
        f.write(a['k'])
    

You can optionally specify the exact encoding you want for the output as an
additional parameter:

    
    
    with io.open('test.txt','w',encoding='utf8') as f:
    

