<p>What is the best practice of Unicode processing in C++?</p>

[{'answer_id': 55644, 'author': 'Adam Pierce', 'author_id': 5324, 'author_profile': 'https://Stackoverflow.com/users/5324', 'pm_score': 3, 'selected': False, 'text': '<p>Here is a checklist for Windows programming:</p>\n\n<ul>\n<li>All strings enclosed in _T("my string")</li>\n<li>strlen() etc. functions replaced with _tcslen() etc.</li>\n<li>Use LPTSTR and LPCTSTR instead of char * and const char *</li>\n<li>When starting new projects in Dev Studio, religiously make sure the Unicode option is selected in your project properties.</li>\n<li>For C++ strings, use std::wstring instead of std::string</li>\n</ul>\n'}, {'answer_id': 55650, 'author': 'hazzen', 'author_id': 5066, 'author_profile': 'https://Stackoverflow.com/users/5066', 'pm_score': 7, 'selected': True, 'text': '<ul>\n<li>Use  <a href="http://www.icu-project.org/" rel="noreferrer">ICU</a> for dealing with your data (or a similar library)</li>\n<li>In your own data store, make sure everything is stored in the same encoding</li>\n<li>Make sure you are always using your unicode library for mundane tasks like string length, capitalization status, etc. Never use standard library builtins like <code>is_alpha</code> unless that is the definition you want.</li>\n<li>I can\'t say it enough: <em><strong>never iterate over the indices of a <code>string</code> if you care about correctness, always use your unicode library for this.</strong></em></li>\n</ul>\n'}, {'answer_id': 55652, 'author': 'Joe Schneider', 'author_id': 1541, 'author_profile': 'https://Stackoverflow.com/users/1541', 'pm_score': 0, 'selected': False, 'text': '<p>Use IBM\'s <a href="http://www.icu-project.org/" rel="nofollow noreferrer">International Components for Unicode</a></p>\n'}, {'answer_id': 55655, 'author': 'ine', 'author_id': 4965, 'author_profile': 'https://Stackoverflow.com/users/4965', 'pm_score': 2, 'selected': False, 'text': '<p>Look at\n<a href="https://stackoverflow.com/questions/11635/case-insensitive-string-comparison-in-c">Case insensitive string comparison in C++</a></p>\n\n<p>That question has a link to the Microsoft documentation on Unicode: <a href="http://msdn.microsoft.com/en-us/library/cc194799.aspx" rel="nofollow noreferrer">http://msdn.microsoft.com/en-us/library/cc194799.aspx</a></p>\n\n<p>If you look on the left-hand navigation side on MSDN next to that article, you should find a lot of information pertaining to Unicode functions. It is part of a chapter on "Encoding Characters" (<a href="http://msdn.microsoft.com/en-us/library/cc194786.aspx" rel="nofollow noreferrer">http://msdn.microsoft.com/en-us/library/cc194786.aspx</a>)</p>\n\n<p>It has the following subsections:</p>\n\n<ul>\n<li>The Code-Page Model</li>\n<li>Double-Byte Character Sets in Windows</li>\n<li>Unicode</li>\n<li>Compatibility Issues in Mixed Environments</li>\n<li>Unicode Data Conversion</li>\n<li>Migrating Windows-Based Programs to Unicode</li>\n<li>Summary</li>\n</ul>\n'}, {'answer_id': 55662, 'author': 'jschroedl', 'author_id': 2420, 'author_profile': 'https://Stackoverflow.com/users/2420', 'pm_score': 3, 'selected': False, 'text': '<p>Our company (and others) use the open source <a href="http://www.icu-project.org/" rel="noreferrer">Internation Components for Unicode</a> (ICU) library originally developed by Taligent.</p>\n\n<p>It handles strings, locales, conversions, date/times, collation, transformations, et. al.</p>\n\n<p>Start with the  <a href="http://www.icu-project.org/userguide/intro.html" rel="noreferrer">ICU Userguide</a></p>\n'}, {'answer_id': 9661638, 'author': 'Willow Schlanger', 'author_id': 1263266, 'author_profile': 'https://Stackoverflow.com/users/1263266', 'pm_score': 2, 'selected': False, 'text': '<p>Although this may not be best practice for everyone, you can write your own C++ UNICODE routines if you want!</p>\n\n<p>I just finished doing it over a weekend. I learned a lot, though I don\'t guarantee it\'s 100% bug free, I did a lot of testing and it seems to work correctly.</p>\n\n<p>My code is under the New BSD license and can be found here:</p>\n\n<p><a href="http://code.google.com/p/netwidecc/downloads/list" rel="nofollow">http://code.google.com/p/netwidecc/downloads/list</a></p>\n\n<p>It is called WSUCONV and comes with a sample main() program that converts between UTF-8, UTF-16, and Standard ASCII. If you throw away the main code, you\'ve got a nice library for reading / writing UNICODE.</p>\n'}, {'answer_id': 13484885, 'author': 'eestrada', 'author_id': 1733321, 'author_profile': 'https://Stackoverflow.com/users/1733321', 'pm_score': 3, 'selected': False, 'text': '<p>If you don\'t care about backwards compatibility with previous C++ standards, the current C++11 standard has built in Unicode support: <a href="http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2011/n3242.pdf" rel="noreferrer">http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2011/n3242.pdf</a></p>\n\n<p>So the truly best practice for Unicode processing in C++ would be to use the built in facilities for it. That isn\'t always a possibility with older code bases though, with the standard being so new at present.</p>\n\n<p>EDIT: To clarify, C++11 is Unicode aware in that it now has support for Unicode literals and Unicode strings. However, the standard library has only <a href="http://www.cplusplus.com/reference/codecvt/" rel="noreferrer">limited support</a> for Unicode processing and conversion. For your current needs this may be enough. However, if you need to do a large amount of heavy lifting right now then you may still need to use something like <a href="http://www.icu-project.org" rel="noreferrer">ICU</a> for more in-depth processing. There are some proposals <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3398.html" rel="noreferrer">currently in the works</a> to include more robust support for text conversion between different encodings. My guess (and hope) is that this will be part of the next <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1810.html" rel="noreferrer">technical report</a>.</p>\n'}, {'answer_id': 14901747, 'author': 'Paul Hutchinson', 'author_id': 511334, 'author_profile': 'https://Stackoverflow.com/users/511334', 'pm_score': 1, 'selected': False, 'text': '<p>As has been said above a library is the best bet when using a large system.  However some times you do want to handle things your self (maybe because the library would use to many resources like on a micro controller).  In this case you want a simple library that you can copy the parts out of for the things you actually need.</p>\n\n<p>Willow Schlanger\'s example code seems like a good one (see his answer for more details).</p>\n\n<p>I also found another one that has smaller code, but lacks full error checking and only handles UTF-8 but was simpler to take parts out of.</p>\n\n<p>Here\'s a list of the embedded libraries that seem decent.</p>\n\n<h3>Embedded libraries</h3>\n\n<ul>\n<li><a href="http://code.google.com/p/netwidecc/downloads/list" rel="nofollow noreferrer">http://code.google.com/p/netwidecc/downloads/list</a> (UTF8, UTF16LE, UTF16BE, UTF32)</li>\n<li><a href="http://www.cprogramming.com/tutorial/unicode.html" rel="nofollow noreferrer">http://www.cprogramming.com/tutorial/unicode.html</a> (UTF8)</li>\n<li><a href="http://utfcpp.sourceforge.net/" rel="nofollow noreferrer">http://utfcpp.sourceforge.net/</a> (Simple UTF8 library)</li>\n</ul>\n'}, {'answer_id': 39657337, 'author': 'Jan Rüegg', 'author_id': 369009, 'author_profile': 'https://Stackoverflow.com/users/369009', 'pm_score': 0, 'selected': False, 'text': '<p>Have a look at the recommendations of <a href="http://utf8everywhere.org/" rel="nofollow">UTF-8 Everywhere</a></p>\n'}]
['https://Stackoverflow.com/questions/55641', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/5694/']