Python unicode string operation in App Engine failing
Question: Python code that runs in Development/local machine, but fails after installing
to Appengine :

1st line in my File :

    
    
    # -*- coding: utf8 -*-O
    

Lines later in the code :

    
    
    s1 = u'Ismerőseid'
    logging.info (s1)
    s2 = s1 + u':' + s1
    logging.info (s2)
    logging.info ("%s,%s", s1, s2)
    

In Dev (localhost):

    
    
    INFO     2012-12-18 04:01:17,926 AppRun.py:662] Ismerőseid,
    INFO     2012-12-18 04:01:17,926 AppRun.py:664] Ismerőseid:Ismerőseid
    INFO     2012-12-18 04:01:17,926 AppRun.py:665] Ismerőseid,Ismerőseid. Ó,
    

On App Engine after install/run :

    
    
    I 2012-12-21 06:52:07.730 
    É, Á, Ö, Ü. Ó,
    
    E 2012-12-21 06:52:07.736
    
    Traceback (most recent call last):
      File "....", line 672, in xxxx
        s3 = s1 + u':' + s1
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128)
    

I have tried to various combination of encoding/decoding/etc.. I have also
chardet on the pasted string 'Ismerőseid' and it gives me `{'confidence':
0.7402600692642154, 'encoding': 'ISO-8859-2'}`

Any help is greatly appreciated!


Answer: Put these 3 lines on the top of your Python 27 code to use unicode :

    
    
    #!/usr/bin/python
    # -*- coding: utf-8 -*-
    from __future__ import unicode_literals
    
    # And this code will not give you any problems
    
    s1 = 'É, Á, Ö, Ü. Ó,'
    logging.info (s1)
    s2 = s1 + ':' + s1
    logging.info ("%s,%s", s1, s2)
    

And never user str(). Only if you realy need to!

And read [this blogpost](http://blog.notdot.net/2010/07/Getting-unicode-right-
in-Python) from Nick Johnson. This was before Python 27. He did not use the
`from __future__ import unicode_literals` , which makes using unicode with
Python so easy.

