<p>When I try to print a Unicode string in a Windows console, I get an error .</p>
<p><code>UnicodeEncodeError: 'charmap' codec can't encode character ....</code></p>
<p>I assume this is because the Windows console does not accept Unicode-only characters. What's the best way around this?
Is there any way I can make Python automatically print a <code>?</code> instead of failing in this situation?</p>
<p><strong>Edit:</strong>  I'm using Python 2.5.</p>
<hr />
<p><strong>Note:</strong> @LasseV.Karlsen answer with the checkmark is sort of outdated (from 2008). Please use the solutions/answers/suggestions below with care!!</p>
<p><a href="https://stackoverflow.com/a/32176732/610569">@JFSebastian answer</a> is more relevant as of today (6 Jan 2016).</p>

[{'answer_id': 5430, 'author': 'Lasse V. Karlsen', 'author_id': 267, 'author_profile': 'https://Stackoverflow.com/users/267', 'pm_score': 6, 'selected': True, 'text': '<p><strong>Note:</strong> This answer is sort of outdated (from 2008). Please use the solution below with care!!</p>\n\n<hr>\n\n<p>Here is a page that details the problem and a solution (search the page for the text <em>Wrapping sys.stdout into an instance</em>):</p>\n\n<p><a href="http://wiki.python.org/moin/PrintFails" rel="noreferrer">PrintFails - Python Wiki</a></p>\n\n<p>Here\'s a code excerpt from that page:</p>\n\n<pre><code>$ python -c \'import sys, codecs, locale; print sys.stdout.encoding; \\\n    sys.stdout = codecs.getwriter(locale.getpreferredencoding())(sys.stdout); \\\n    line = u"\\u0411\\n"; print type(line), len(line); \\\n    sys.stdout.write(line); print line\'\n  UTF-8\n  &lt;type \'unicode\'&gt; 2\n  Б\n  Б\n\n  $ python -c \'import sys, codecs, locale; print sys.stdout.encoding; \\\n    sys.stdout = codecs.getwriter(locale.getpreferredencoding())(sys.stdout); \\\n    line = u"\\u0411\\n"; print type(line), len(line); \\\n    sys.stdout.write(line); print line\' | cat\n  None\n  &lt;type \'unicode\'&gt; 2\n  Б\n  Б\n</code></pre>\n\n<p>There\'s some more information on that page, well worth a read.</p>\n'}, {'answer_id': 8020, 'author': 'Bartosz Radaczyński', 'author_id': 985, 'author_profile': 'https://Stackoverflow.com/users/985', 'pm_score': 1, 'selected': False, 'text': '<p>The cause of your problem is <strong>NOT</strong> the Win console not willing to accept Unicode (as it does this since I guess Win2k by default). It is the default system encoding. Try this code and see what it gives you:</p>\n<pre><code>import sys\nsys.getdefaultencoding()\n</code></pre>\n<p>if it says ascii, there\'s your cause ;-)\nYou have to create a file called sitecustomize.py and put it under python path (I put it under /usr/lib/python2.5/site-packages, but that is differen on Win - it is c:\\python\\lib\\site-packages or something), with the following contents:</p>\n<pre><code>import sys\nsys.setdefaultencoding(\'utf-8\')\n</code></pre>\n<p>and perhaps you might want to specify the encoding in your files as well:</p>\n<pre><code># -*- coding: UTF-8 -*-\nimport sys,time\n</code></pre>\n<p>Edit: more info can be found <a href="http://web.archive.org/web/20120315050914/http://www.diveintopython.net/xml_processing/unicode.html" rel="nofollow noreferrer">in excellent the Dive into Python book</a></p>\n'}, {'answer_id': 2013263, 'author': 'sorin', 'author_id': 99834, 'author_profile': 'https://Stackoverflow.com/users/99834', 'pm_score': 3, 'selected': False, 'text': '<p>The below code will make Python output to console as UTF-8 even on Windows. </p>\n\n<p>The console will display the characters well on Windows 7 but on Windows XP it will not display them well, but at least it will work and most important you will have a consistent output from your script on all platforms. You\'ll be able to redirect the output to a file.</p>\n\n<p>Below code was tested with Python 2.6 on Windows.</p>\n\n<pre><code>\n#!/usr/bin/python\n# -*- coding: UTF-8 -*-\n\nimport codecs, sys\n\nreload(sys)\nsys.setdefaultencoding(\'utf-8\')\n\nprint sys.getdefaultencoding()\n\nif sys.platform == \'win32\':\n    try:\n        import win32console \n    except:\n        print "Python Win32 Extensions module is required.\\n You can download it from https://sourceforge.net/projects/pywin32/ (x86 and x64 builds are available)\\n"\n        exit(-1)\n    # win32console implementation  of SetConsoleCP does not return a value\n    # CP_UTF8 = 65001\n    win32console.SetConsoleCP(65001)\n    if (win32console.GetConsoleCP() != 65001):\n        raise Exception ("Cannot set console codepage to 65001 (UTF-8)")\n    win32console.SetConsoleOutputCP(65001)\n    if (win32console.GetConsoleOutputCP() != 65001):\n        raise Exception ("Cannot set console output codepage to 65001 (UTF-8)")\n\n#import sys, codecs\nsys.stdout = codecs.getwriter(\'utf8\')(sys.stdout)\nsys.stderr = codecs.getwriter(\'utf8\')(sys.stderr)\n\nprint "This is an Е乂αmp١ȅ testing Unicode support using Arabic, Latin, Cyrillic, Greek, Hebrew and CJK code points.\\n"\n</code></pre>\n'}, {'answer_id': 4637795, 'author': 'Daira Hopwood', 'author_id': 393146, 'author_profile': 'https://Stackoverflow.com/users/393146', 'pm_score': 5, 'selected': False, 'text': '<p><strong>Update:</strong> On Python 3.6 or later, printing Unicode strings to the console on Windows just works.</p>\n<p>So, upgrade to recent Python and you\'re done. At this point I recommend using 2to3 to update your code to Python 3.x if needed, and just dropping support for Python 2.x. Note that there has been no security support for any version of Python before 3.7 (including Python 2.7) <a href="https://endoflife.date/python" rel="nofollow noreferrer">since December 2021</a>.</p>\n<p>If you <em>really</em> still need to support earlier versions of Python (including Python 2.7), you can use <a href="https://github.com/Drekin/win-unicode-console" rel="nofollow noreferrer">https://github.com/Drekin/win-unicode-console</a> , which is based on, and uses the same APIs as the code in the answer that was <a href="https://stackoverflow.com/a/3259271/393146">previously linked here</a>. (That link does include some information on Windows font configuration but I doubt it still applies to Windows 8 or later.)</p>\n<p>Note: despite other plausible-sounding answers that suggest changing the code page to 65001, that <a href="https://github.com/python/cpython/issues/45943" rel="nofollow noreferrer">did not work prior to Python 3.8</a>. (It does kind-of work since then, but as pointed out above, you don\'t need to do so for Python 3.6+ anyway.) Also, changing the default encoding using <code>sys.setdefaultencoding</code> is (still) <a href="https://stackoverflow.com/questions/3578685/how-to-display-utf-8-in-windows-console/3580165#3580165">not a good idea</a>.</p>\n'}, {'answer_id': 10667978, 'author': 'Giampaolo Rodolà', 'author_id': 376587, 'author_profile': 'https://Stackoverflow.com/users/376587', 'pm_score': 4, 'selected': False, 'text': '<p>If you\'re not interested in getting a reliable representation of the bad character(s) you might use something like this (working with python >= 2.6, including 3.x):</p>\n\n<pre><code>from __future__ import print_function\nimport sys\n\ndef safeprint(s):\n    try:\n        print(s)\n    except UnicodeEncodeError:\n        if sys.version_info &gt;= (3,):\n            print(s.encode(\'utf8\').decode(sys.stdout.encoding))\n        else:\n            print(s.encode(\'utf8\'))\n\nsafeprint(u"\\N{EM DASH}")\n</code></pre>\n\n<p>The bad character(s) in the string will be converted in a representation which is printable by the Windows console.</p>\n'}, {'answer_id': 32176732, 'author': 'jfs', 'author_id': 4279, 'author_profile': 'https://Stackoverflow.com/users/4279', 'pm_score': 6, 'selected': False, 'text': '<p><strong>Update:</strong> <a href="https://docs.python.org/3.6/whatsnew/3.6.html#pep-528-change-windows-console-encoding-to-utf-8" rel="noreferrer">Python 3.6</a> implements <a href="https://www.python.org/dev/peps/pep-0528/" rel="noreferrer">PEP 528: Change Windows console encoding to UTF-8</a>: <em>the default console on Windows will now accept all Unicode characters.</em> Internally, it uses the same Unicode API as <a href="https://github.com/Drekin/win-unicode-console" rel="noreferrer">the <code>win-unicode-console</code> package mentioned below</a>. <code>print(unicode_string)</code> should just work now.</p>\n<hr />\n<blockquote>\n<p>I get a <code>UnicodeEncodeError: \'charmap\' codec can\'t encode character...</code>  error.</p>\n</blockquote>\n<p>The error means that Unicode characters that you are trying to print can\'t be represented using the current (<code>chcp</code>) console character encoding. The codepage is often 8-bit encoding such as <code>cp437</code> that can represent only ~0x100 characters from ~1M Unicode characters:</p>\n<pre>>>> u"\\N{EURO SIGN}".encode(\'cp437\')\nTraceback (most recent call last):\n...\nUnicodeEncodeError: \'charmap\' codec can\'t encode character \'\\u20ac\' in position 0:\ncharacter maps to </pre>\n<blockquote>\n<p>I assume this is because the Windows console does not accept Unicode-only characters. What\'s the best way around this?</p>\n</blockquote>\n<p>Windows console does accept Unicode characters and it can even display them (BMP only) <strong>if the corresponding font is configured</strong>. <code>WriteConsoleW()</code> API should be used as suggested in <a href="https://stackoverflow.com/a/4637795/4279">@Daira Hopwood\'s answer</a>. It can be called transparently i.e., you don\'t need to and should not modify your scripts if you use <a href="https://github.com/Drekin/win-unicode-console" rel="noreferrer"><code>win-unicode-console</code> package</a>:</p>\n<pre><code>T:\\&gt; py -m pip install win-unicode-console\nT:\\&gt; py -m run your_script.py\n</code></pre>\n<p>See <a href="https://stackoverflow.com/a/30551552/4279">What\'s the deal with Python 3.4, Unicode, different languages and Windows?</a></p>\n<blockquote>\n<p>Is there any way I can make Python\nautomatically print a <code>?</code> instead of failing in this situation?</p>\n</blockquote>\n<p>If it is enough to replace all unencodable characters with <code>?</code> in your case then you could set <a href="https://docs.python.org/3/using/cmdline.html#envvar-PYTHONIOENCODING" rel="noreferrer"><code>PYTHONIOENCODING</code> envvar</a>:</p>\n<pre><code>T:\\&gt; set PYTHONIOENCODING=:replace\nT:\\&gt; python3 -c &quot;print(u\'[\\N{EURO SIGN}]\')&quot;\n[?]\n</code></pre>\n<p>In Python 3.6+, the encoding specified by <code>PYTHONIOENCODING</code> envvar is ignored for interactive console buffers unless <code>PYTHONLEGACYWINDOWSIOENCODING</code> envvar is set to a non-empty string.</p>\n'}, {'answer_id': 34306583, 'author': 'Kinjal Dixit', 'author_id': 6629, 'author_profile': 'https://Stackoverflow.com/users/6629', 'pm_score': 2, 'selected': False, 'text': '<p>Kind of related on the answer by J. F. Sebastian, but more direct.</p>\n\n<p>If you are having this problem when printing to the console/terminal, then do this:</p>\n\n<pre><code>&gt;set PYTHONIOENCODING=UTF-8\n</code></pre>\n'}, {'answer_id': 35903736, 'author': 'mike rodent', 'author_id': 595305, 'author_profile': 'https://Stackoverflow.com/users/595305', 'pm_score': 3, 'selected': False, 'text': '<p>Like Giampaolo Rodolà\'s answer, but even more dirty: I really, really intend to spend a long time (soon) understanding the whole subject of encodings and how they apply to Windoze consoles, </p>\n\n<p>For the moment I just wanted sthg which would mean my program would NOT CRASH, and which I understood ... and also which didn\'t involve importing too many exotic modules (in particular I\'m using Jython, so half the time a Python module turns out not in fact to be available).</p>\n\n<pre><code>def pr(s):\n    try:\n        print(s)\n    except UnicodeEncodeError:\n        for c in s:\n            try:\n                print( c, end=\'\')\n            except UnicodeEncodeError:\n                print( \'?\', end=\'\')\n</code></pre>\n\n<p>NB "pr" is shorter to type than "print" (and quite a bit shorter to type than "safeprint")...!</p>\n'}, {'answer_id': 37229972, 'author': 'CODE-REaD', 'author_id': 5025060, 'author_profile': 'https://Stackoverflow.com/users/5025060', 'pm_score': -1, 'selected': False, 'text': "<p>James Sulak asked,</p>\n\n<blockquote>\n  <p>Is there any way I can make Python automatically print a ? instead of failing in this situation?</p>\n</blockquote>\n\n<p>Other solutions recommend we attempt to modify the Windows environment or replace Python's <code>print()</code> function.  The answer below comes closer to fulfilling Sulak's request.</p>\n\n<p>Under Windows 7, Python 3.5 can be made to print Unicode without throwing a <code>UnicodeEncodeError</code> as follows:</p>\n\n<p>&nbsp; &nbsp; In place of:\n&nbsp; &nbsp;<code>print(text)</code><br>\n&nbsp; &nbsp; substitute:\n&nbsp; &nbsp; <code>print(str(text).encode('utf-8'))</code></p>\n\n<p>Instead of throwing an exception, Python now displays unprintable Unicode characters as <em>\\xNN</em> hex codes, e.g.:</p>\n\n<p>&nbsp; <em>Halmalo n\\xe2\\x80\\x99\\xc3\\xa9tait plus qu\\xe2\\x80\\x99un point noir</em></p>\n\n<p>Instead of</p>\n\n<p>&nbsp; <em>Halmalo n’était plus qu’un point noir</em></p>\n\n<p>Granted, the latter is preferable <em>ceteris paribus</em>, but otherwise the former is completely accurate for diagnostic messages.  Because it displays Unicode as literal byte values the former may also assist in diagnosing encode/decode problems.</p>\n\n<p><strong>Note:</strong> The <code>str()</code> call above is needed because otherwise <code>encode()</code> causes Python to reject a Unicode character as a tuple of numbers.</p>\n"}, {'answer_id': 43924648, 'author': 'J. Does', 'author_id': 7227370, 'author_profile': 'https://Stackoverflow.com/users/7227370', 'pm_score': 1, 'selected': False, 'text': "<p>Python 3.6 windows7: There is several way to launch a python you could use the python console (which has a python logo on it) or the windows console (it's written cmd.exe on it). </p>\n\n<p>I could not print utf8 characters in the windows console. Printing utf-8 characters throw me this error:</p>\n\n<pre><code>OSError: [winError 87] The paraneter is incorrect \nException ignored in: (_io-TextIOwrapper name='(stdout)' mode='w' ' encoding='utf8') \nOSError: [WinError 87] The parameter is incorrect \n</code></pre>\n\n<p>After trying and failing to understand the answer above I discovered it was only a setting problem. Right click on the top of the cmd console windows, on the tab <code>font</code> chose lucida console.</p>\n"}, {'answer_id': 45868162, 'author': 'shubaly', 'author_id': 7269358, 'author_profile': 'https://Stackoverflow.com/users/7269358', 'pm_score': 2, 'selected': False, 'text': '<p>For Python 2 try:</p>\n\n<pre><code>print unicode(string, \'unicode-escape\')\n</code></pre>\n\n<p>For Python 3 try:</p>\n\n<pre><code>import os\nstring = "002 Could\'ve Would\'ve Should\'ve"\nos.system(\'echo \' + string)\n</code></pre>\n\n<p>Or try win-unicode-console:</p>\n\n<pre><code>pip install win-unicode-console\npy -mrun your_script.py\n</code></pre>\n'}, {'answer_id': 51124805, 'author': 'Matthew Estock', 'author_id': 2133918, 'author_profile': 'https://Stackoverflow.com/users/2133918', 'pm_score': 2, 'selected': False, 'text': "<p>TL;DR:</p>\n<pre><code>print(yourstring.encode('ascii','replace').decode('ascii'))\n</code></pre>\n<hr />\n<p>I ran into this myself, working on a Twitch chat (IRC) bot. (Python 2.7 latest)</p>\n<p>I wanted to parse chat messages in order to respond...</p>\n<pre><code>msg = s.recv(1024).decode(&quot;utf-8&quot;)\n</code></pre>\n<p>but also print them safely to the console in a human-readable format:</p>\n<pre><code>print(msg.encode('ascii','replace').decode('ascii'))\n</code></pre>\n<p>This corrected the issue of the bot throwing <code>UnicodeEncodeError: 'charmap'</code> errors and replaced the unicode characters with <code>?</code>.</p>\n"}, {'answer_id': 52617143, 'author': 'c97', 'author_id': 397961, 'author_profile': 'https://Stackoverflow.com/users/397961', 'pm_score': 3, 'selected': False, 'text': '<p>Just enter this code in command line before executing python script:</p>\n\n<pre><code>chcp 65001 &amp; set PYTHONIOENCODING=utf-8\n</code></pre>\n'}, {'answer_id': 68507839, 'author': 'Itachi', 'author_id': 1831784, 'author_profile': 'https://Stackoverflow.com/users/1831784', 'pm_score': -1, 'selected': False, 'text': '<p>The issue is with windows default encoding being set to cp1252, and need to be set to utf-8. (<a href="https://discuss.python.org/t/pep-597-use-utf-8-for-default-text-file-encoding/1819" rel="nofollow noreferrer">check PEP</a>)</p>\n<p>Check default encoding using:</p>\n<pre><code>import locale \nlocale.getpreferredencoding()\n</code></pre>\n<p>You can override locale settings</p>\n<pre><code>import os\nif os.name == &quot;nt&quot;:\n    import _locale\n    _locale._gdl_bak = _locale._getdefaultlocale\n    _locale._getdefaultlocale = (lambda *args: (_locale._gdl_bak()[0], \'utf8\'))\n</code></pre>\n<p>referenced code from <a href="https://stackoverflow.com/questions/31469707/changing-the-locale-preferred-encoding-in-python-3-in-windows/34345136#34345136">stack link</a></p>\n'}, {'answer_id': 74509356, 'author': 'Wok', 'author_id': 376454, 'author_profile': 'https://Stackoverflow.com/users/376454', 'pm_score': 0, 'selected': False, 'text': '<p>Nowadays, the Windows console does not encounter this error, <strong>unless you redirect the output</strong>.</p>\n<p>Here is an example Python script <code>scratch_1.py</code>:</p>\n<pre class="lang-py prettyprint-override"><code>s = &quot;∞&quot;\n\nprint(s)\n</code></pre>\n<p>If you run the script as follows, everything works as intended:</p>\n<pre class="lang-bash prettyprint-override"><code>python scratch_1.py\n</code></pre>\n<pre><code>∞\n</code></pre>\n<p>However, if you run the following, then you get the same error as in the question:</p>\n<pre class="lang-bash prettyprint-override"><code>python scratch_1.py &gt; temp.txt\n</code></pre>\n<pre><code>Traceback (most recent call last):\n  File &quot;C:\\Users\\Wok\\AppData\\Roaming\\JetBrains\\PyCharmCE2022.2\\scratches\\scratch_1.py&quot;, line 3, in &lt;module&gt;\n    print(s)\n  File &quot;C:\\Users\\Wok\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\encodings\\cp1252.py&quot;, line 19, in encode\n    return codecs.charmap_encode(input,self.errors,encoding_table)[0]\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nUnicodeEncodeError: \'charmap\' codec can\'t encode character \'\\u221e\' in position 0: character maps to &lt;undefined&gt;\n</code></pre>\n<hr />\n<p>To solve this issue with the suggestion present in the original question, i.e. by replacing the erroneous characters with question marks <code>?</code>, one can proceed as follows:</p>\n<pre class="lang-py prettyprint-override"><code>s = &quot;∞&quot;\n\ntry:\n    print(s)\nexcept UnicodeEncodeError:\n    output_str = s.encode(&quot;ascii&quot;, errors=&quot;replace&quot;).decode(&quot;ascii&quot;)\n\n    print(output_str)\n</code></pre>\n<p>It is important:</p>\n<ul>\n<li>to call <code>decode()</code>, so that the type of the output is <code>str</code> instead of <code>bytes</code>,</li>\n<li>with the same encoding, here <code>&quot;ascii&quot;</code>, to avoid the creation of <a href="https://en.wikipedia.org/wiki/Mojibake" rel="nofollow noreferrer">mojibake</a>.</li>\n</ul>\n'}]
['https://Stackoverflow.com/questions/5419', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/207/']