title: character encoding in python
body: <p>I have a byte stream that looks like this '\xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e'</p>
  
  <pre><code>str_data = '\xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e'
  </code></pre>
  
  <p>str_data is wrote into text file using the following code</p>
  
  <pre><code>file = open("test_doc","w")
  file.write(str_data)
  file.close()
  </code></pre>
  
  <p>If test_doc is opened in a web browser and character encoding is set to Japanese it just works fine.</p>
  
  <p>I am using reportlab for generating pdf . using the following code</p>
  
  <pre><code>from reportlab.pdfbase import pdfmetrics
  from reportlab.pdfgen.canvas import Canvas
  from reportlab.pdfbase.cidfonts import CIDFont
  
  
  pdfmetrics.registerFont(CIDFont('HeiseiMin-W3','90ms-RKSJ-H'))
  pdfmetrics.registerFont(CIDFont('HeiseiKakuGo-W5','90ms-RKSJ-H'))
  c = Canvas('test1.pdf')
  c.setFont('HeiseiMin-W3-90ms-RKSJ-H', 6)
  
  message1 = '\202\261\202\352\202\315\225\275\220\254\226\276\222\251\202\305\202\267\201B'
  
  message3 = '\xe3\x83\x86\xe3\x82\xb9\xe3\x83\x88';
  
  c.drawString(100, 675,message1)
  c.save()
  </code></pre>
  
  <p>Here I use message1 variable which gives output in Japanese I need to use message3 instead of message1 to generate the pdf. message3 generated garabage probably because of improper encoding. </p>