title: Problems with Python in Google App Engine - UTF-8 and ASCII
body: <p>So for the past few days I've been trying to learn Python in App Engine. However, I've been encountering a number of problems with ASCII and UTF encoding. The freshest issue is as follows:</p>
  
  <p>I have the following piece of code of a simplistic chatroom from the book 'Code in the Cloud'</p>
  
  <pre><code>from google.appengine.ext import webapp
  from google.appengine.ext.webapp.util import run_wsgi_app
  import datetime
  
  
  # START: MainPage
  class ChatMessage(object):
  def __init__(self, user, msg):
      self.user = user
      self.message = msg
      self.time = datetime.datetime.now()
  
  def __str__(self):
      return "%s (%s): %s" % (self.user, self.time, self.message)
  
  Messages = []
  
  class ChatRoomPage(webapp.RequestHandler):
  def get(self):
      self.response.headers["Content-Type"] = "text/html"
      self.response.out.write("""
         &lt;html&gt;
           &lt;head&gt;
             &lt;title&gt;MarkCC's AppEngine Chat Room&lt;/title&gt;
           &lt;/head&gt;
           &lt;body&gt;
             &lt;h1&gt;Welcome to MarkCC's AppEngine Chat Room&lt;/h1&gt;
             &lt;p&gt;(Current time is %s)&lt;/p&gt;
         """ % (datetime.datetime.now()))
      # Output the set of chat messages
      global Messages
      for msg in Messages:
          self.response.out.write("&lt;p&gt;%s&lt;/p&gt;" % msg)
      self.response.out.write("""
         &lt;form action="" method="post"&gt;
         &lt;div&gt;&lt;b&gt;Name:&lt;/b&gt; 
         &lt;textarea name="name" rows="1" cols="20"&gt;&lt;/textarea&gt;&lt;/div&gt;
         &lt;p&gt;&lt;b&gt;Message&lt;/b&gt;&lt;/p&gt;
         &lt;div&gt;&lt;textarea name="message" rows="5" cols="60"&gt;&lt;/textarea&gt;&lt;/div&gt;
         &lt;div&gt;&lt;input type="submit" value="Send ChatMessage"&gt;&lt;/input&gt;&lt;/div&gt;
         &lt;/form&gt;
       &lt;/body&gt;
     &lt;/html&gt;
     """)
   # END: MainPage    
   # START: PostHandler
  def post(self):
      chatter = self.request.get("name")
      msg = self.request.get("message")
      global Messages
      Messages.append(ChatMessage(chatter, msg))
      # Now that we've added the message to the chat, we'll redirect
      # to the root page, which will make the user's browser refresh to
      # show the chat including their new message.
      self.redirect('/')        
  # END: PostHandler
  
  
  
  
  # START: Frame
  chatapp = webapp.WSGIApplication([('/', ChatRoomPage)])
  
  
  def main():
  run_wsgi_app(chatapp)
  
  if __name__ == "__main__":
  main()
  # END: Frame
  </code></pre>
  
  <p>It works ok in English. However, the moment I add some non-standard characters all sorts of problems start</p>
  
  <p>First of all, in order for the thing to be actually able to display characters in HTML I add meta tag - charset=UTF-8" etc</p>
  
  <p>Curiously, if you enter non-standard letters, the program processes them nicely, and displays them with no issues. However, it fails to load if I enter any non-ascii letters to the web layout iteself withing the script. I figured out that adding utf-8 encoding line would work. So I added (# -<em>- coding: utf-8 -</em>-). This was not enough. Of course I forgot to save the file in UTF-8 format. Upon that the program started running. </p>
  
  <p>That would be the good end to the story, alas....</p>
  
  <p>It doesn't work</p>
  
  <p>Long story short this code:</p>
  
  <pre><code># -*- coding: utf-8 -*-
  from google.appengine.ext import webapp
  from google.appengine.ext.webapp.util import run_wsgi_app
  import datetime
  
  
  # START: MainPage
  class ChatMessage(object):
  def __init__(self, user, msg):
      self.user = user
      self.message = msg
      self.time = datetime.datetime.now()
  
  def __str__(self):
      return "%s (%s): %s" % (self.user, self.time, self.message)
  
  Messages = []
  class ChatRoomPage(webapp.RequestHandler):
  def get(self):
      self.response.headers["Content-Type"] = "text/html"
      self.response.out.write("""
         &lt;html&gt;
           &lt;head&gt;
             &lt;title&gt;Witaj w pokoju czatu MarkCC w App Engine&lt;/title&gt;
             &lt;meta http-equiv="Content-Type" content="text/html; charset=UTF-8"&gt;
           &lt;/head&gt;
           &lt;body&gt;
             &lt;h1&gt;Witaj w pokoju czatu MarkCC w App Engine&lt;/h1&gt;
             &lt;p&gt;(Dokladny czas Twojego logowania to: %s)&lt;/p&gt;
         """ % (datetime.datetime.now()))
      # Output the set of chat messages
      global Messages
      for msg in Messages:
          self.response.out.write("&lt;p&gt;%s&lt;/p&gt;" % msg)
      self.response.out.write("""
         &lt;form action="" method="post"&gt;
         &lt;div&gt;&lt;b&gt;Twój Nick:&lt;/b&gt; 
         &lt;textarea name="name" rows="1" cols="20"&gt;&lt;/textarea&gt;&lt;/div&gt;
         &lt;p&gt;&lt;b&gt;Twoja Wiadomość&lt;/b&gt;&lt;/p&gt;
         &lt;div&gt;&lt;textarea name="message" rows="5" cols="60"&gt;&lt;/textarea&gt;&lt;/div&gt;
         &lt;div&gt;&lt;input type="submit" value="Send ChatMessage"&gt;&lt;/input&gt;&lt;/div&gt;
         &lt;/form&gt;
       &lt;/body&gt;
     &lt;/html&gt;
     """)
  # END: MainPage    
  # START: PostHandler
  def post(self):
      chatter = self.request.get(u"name")
      msg = self.request.get(u"message")
      global Messages
      Messages.append(ChatMessage(chatter, msg))
      # Now that we've added the message to the chat, we'll redirect
      # to the root page, which will make the user's browser refresh to
      # show the chat including their new message.
      self.redirect('/')        
  # END: PostHandler
  
  
  
  
  # START: Frame
  chatapp = webapp.WSGIApplication([('/', ChatRoomPage)])
  
  
  def main():
  run_wsgi_app(chatapp)
  
  if __name__ == "__main__":
  main()
  # END: Frame
  </code></pre>
  
  <p>Fails to process anything I write in the chat application when it's running. It loads but the moment I enter my message (even using only standard characters) I receive </p>
  
  <pre><code>File "D:\Python25\lib\StringIO.py", line 270, in getvalue
  self.buf += ''.join(self.buflist)
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 64: ordinal not in       range(128) 
  </code></pre>
  
  <p>error message. In other words, if I want to be able to use any characters within the application I cannot put non-English ones in my interface. Or the other way round, I can use non-English characters within the app only if I don't encode the file in utf-8. How to make it all work together?</p>