Python CSV Module Issue With JSON
Question: I am using the python `csv` module to create a csv where some of the values
are `json` strings. However the `csv` module's quoting is totally breaking the
json:

    
    
    import csv
    import json
    writer = csv.writer(open('tmp', 'w'))
    writer.writerow([json.dumps([{'a' : 'b'}])])
    

The outputted json is broken, as you can see:

    
    
    cat tmp
    > "[{""a"": ""b""}]"
    
    import json
    json.loads("[{""a"": ""b""}]")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
        return _default_decoder.decode(s)
      File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
        obj, end = self.scan_once(s, idx)
    ValueError: Expecting property name: line 1 column 2 (char 2)
    

And `csv` objects to turning quoting off:

    
    
    import csv
    import json
    writer = csv.writer(open('tmp', 'w'), quoting=csv.QUOTE_NONE)
    writer.writerow([json.dumps([{u'a' : u'b'}])])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    _csv.Error: need to escape, but no escapechar set
    

Has anyone else encountered this? Do `json` and `csv` just not play well
together? (It's not my idea to store `json` stirngs in `csv` files.. something
I just need to deal with right now). Unfortunately, these csvs I am creating
contain hash digests and all sorts of other complicated stuff so all the `sed`
or `awk`ish type solutions to fix the json I've tried have failed or messed up
something else..


Answer: don't use `"` as your quote character. Use something else:

    
    
    with open('tmp', 'w') as fout:
        writer = csv.writer(fout, quotechar="'")
    

Really, this just tidy's things up a bit. When you read the data back in, you
first need to "unquote" it by reading the data via `csv.reader`. That should
give you back the strings you put in which are valid `json`.

