issue with converting html entities and encoding
<p>I'm using <a href="http://effbot.org/zone/re-sub.htm#unescape-html" rel="nofollow">this function</a> to escape the HTML enities</p>

<pre><code>import re, htmlentitydefs

##
# Removes HTML or XML character references and entities from a text string.
#
# @param text The HTML (or XML) source text.
# @return The plain text, as a Unicode string, if necessary.

def unescape(text):
    def fixup(m):
        text = m.group(0)
        if text[:2] == "&amp;#":
            # character reference
            try:
                if text[:3] == "&amp;#x":
                    return unichr(int(text[3:-1], 16))
                else:
                    return unichr(int(text[2:-1]))
            except ValueError:
                pass
        else:
            # named entity
            try:
                text = unichr(htmlentitydefs.name2codepoint[text[1:-1]])
            except KeyError:
                pass
        return text # leave as is
    return re.sub("&amp;#?\w+;", fixup, text)
</code></pre>

<p>but when i try to process some text i get this error, (most of the text works) but python throws me this error</p>

<pre><code>File "C:\Python27\lib\encodings\cp437.py", line 12, in encode
  return codecs.charmap_encode(input,errors,encoding_map)
  UnicodeEncodeError: 'charmap' codec can't encode character u'\xae' in position 3
 48: character maps to &lt;undefined&gt;
</code></pre>

<p>i have tried encoding the text string a million different ways, nothing is working so far ascii, utf, unicode... all that stuff which i really don't understand</p>
