title: C++ unicode questions
body: <p>I'm aware of ICU and small libraries like the utf8 one on code project (forget the exact name) however none of these are exactly what I want.</p>
  
  <p>What I really want is something like ICU but wrapped up in a more friendly manner.</p>
  
  <p>Specifically:</p>
  
  <ul>
  <li>Fully Object Orientated  </li>
  <li>Implementations of the c++ standard streams, or at least something that performs the same role.  </li>
  <li>Can format time, dates etc in a locale dependent manner (eg dd/mm/yy in the UK and mm/dd/yy in the US).  </li>
  <li>Lets me choose the "internal" encoding of strings, so I can for example make it use UTF-16 on windows to avoid lots of conversions when passing strings to and from the windows API and DirectX  </li>
  <li>Easy converting of strings between encodings  </li>
  </ul>
  
  <p>If no such library exists, is it possible to wrap the ICU up using the standard c++ classes, so I can for example create a ustring which has identical usage to std::string and std::wstring, and also implement versions of the streams (optimally with them being fully compatible with the existing ones, ie I could pass it to a function expecting an std::ostream and it will perform conversion between its internal format and ascii (or utf-8) on the fly)? Assuming it is possible just how much work would it be?</p>
  
  <p>EDIT:
  Also having looked at the c++0x standard and noticed literals for utf8, utf16 and utf32, does that mean that standard library (eg strings, streams, etc) will fully support those encodeings and the conversion between them? If so anyone got any idea how long it will be until Visual Studio will support those features?</p>
  
  <p>EDIT2:
  As for using the existing c++ support, I'll look up the locale and facet stuff.</p>
  
  <p>One of the problems I ran into is that when using streams defined around wchar_t which is 2 bytes under windows for file i/o however is it still seemed to use ascii for the files them selves.</p>
  
  <pre><code>std::wofstream file(L"myfile.txt", std::ios::out);
  file &lt;&lt; L"Hello World!" &lt;&lt; std::endl;
  </code></pre>
  
  <p>resulted in the following hex in the file<br>
  48 65 6C 6C 6F 20 57 6F 72 6C 64 0D 0A<br>
  which is clearly ascii rather than the expected utf-16 output of:<br>
  FF FE 48 00 65 00 6C 00 6C 00 6F 00 20 00 57 00 6F 00 72 00 6C 00 64 00 0D 00 0A 00</p>