Python Character Encoding?
<p>I'm not sure with the title I'm used, but basically I want to get rid off weird characters from the string. Here is the code:</p>

<pre><code>#!/usr/bin/python
# -*- coding: utf-8 -*-
# source: http://snippets.dzone.com/posts/show/4569

from htmlentitydefs import name2codepoint as n2cp
import re

def substitute_entity(match):
    ent = match.group(3)

    if match.group(1) == "#":
        if match.group(2) == '':
            return unichr(int(ent))
        elif match.group(2) == 'x':
            return unichr(int('0x'+ent, 16))
    else:
        cp = n2cp.get(ent)

        if cp:
           return unichr(cp)
        else:
           return match.group()

def decode_htmlentities(string):
    entity_re = re.compile(r'&amp;(#?)(x?)(\d{1,5}|\w{1,8});')
    return entity_re.subn(substitute_entity, string)[0]



test = ['&lt;b&gt;Blogger&lt;/b&gt; in the Classroom - &lt;b&gt;Google&lt;/b&gt;', 'Of\xef\xac\x81cial &lt;b&gt;Google   Blog&lt;/b&gt;']
container = []

for i in test:
    container.append(decode_htmlentities(i))
print container

for i in test:
    print decode_htmlentities(i)
</code></pre>

<p>And here is the result:</p>

<pre><code>['&lt;b&gt;Blogger&lt;/b&gt; in the Classroom - &lt;b&gt;Google&lt;/b&gt;', 'Of\xef\xac\x81cial &lt;b&gt;Google Blog&lt;/b&gt;']

&lt;b&gt;Blogger&lt;/b&gt; in the Classroom - &lt;b&gt;Google&lt;/b&gt;
Ofﬁcial &lt;b&gt;Google Blog&lt;/b&gt;
</code></pre>

<p>The Question:</p>

<p>Using the same function (decode_htmlentities()), why I get a different result when appending into a list and 'just' printing?</p>

<p>Here is the difference:</p>

<pre><code>Of\xef\xac\x81cial &lt;b&gt;Google Blog&lt;/b&gt; # output from list
Ofﬁcial &lt;b&gt;Google Blog&lt;/b&gt; # output from print 
</code></pre>
