u'Too' u'much' u'unicode' u'returned'
Question: I have an api which I'm putting things into and out of in a natural language
processing context, using json.

_Everything_ is coming out as unicode. For example, if retrieve a list of
words from my api, every single word is u''. This is what the json output
looks like after printing to a file:

    
    
    {u'words': [u'every', u'single', u'word']}
    

I must clarify that in the terminal everything looks good, just not when I
print the output to a file.

I haven't figured out yet if this is preferable default behavior or if I need
to do something along the way to make this plain, or what. The outputs are
going to used with languages other than python, other contexts where they need
to be readable and/or parseable.

So clearly I don't have a grasp on python & unicode and how and where this is
being.

  1. Is this preferable when dealing with json? Should I not worry about it?

  2. How I turn this off, or how do I take an extra step (I've already tried but can't figure out exactly where this is doing this) to make this less of a nuisance.

I have much to learn, so any input is appreciated.

EDIT: all the input has been useful, thank you.

I was under the mistaken notion that jsonify did more than it actually does I
guess. If I do json.dumps earlier in my task chain, I get actual json on the
other end.


Answer: There is nothing wrong with this, and you don't need to do anything about it.

In Python 2, a `str` is similar to a C string - it's just a sequence of bytes,
sometimes incorrectly assumed to be ASCII text. It can contain _encoded_ text,
e.g. as UTF-8 or ASCII.

The `unicode` type represents an actual string of text, similar to a Java
`String`. It is text in the abstract sense, not tied to a particular encoding.
You can decode a `str` into `unicode`, or encode a `unicode` into a `str`.

JSON keys and values are strings - they are not byte arrays, but text - so
they are represented by `unicode` objects in Python.

If you need JSON output for use in another language, use the `json` module to
produce it from your dictionary:

    
    
    >>> import json
    >>> print json.dumps({u'words': [u'every', u'single', u'word']})
    {"words": ["every", "single", "word"]}
    

