Issues with designing an API for a UTF String class
<p>I've been making an UTF string class for a while now, which can process UTF8/UTF16/UTF32, and also can convert it's data into those above. I have also defined few types: <code>char8, char16, char32</code> which are based on <code>unsigned char, unsigned short/wchar_t, unsigned int/wchar_t</code>, this way char16* and char32* is compatible with <code>wchar_t*</code> and also with calls taking that type. Whether the <code>char16</code> or <code>char32</code> is based on <code>wchar_t</code> depends on the platform cause some platform use 32 bits sized <code>wchar_t</code>, others use 16 bits sized one.</p>

<p>The internal storage uses raw form - integer arrays stored in chunks - so I also needed some sort of encode call where I output data in 8/16/32 unit based encoding. So I was going to make overload for all encode types:</p>

<pre><code>U32 UTF::encode( char8** Out );
U32 UTF::encode( char16** Out );
U32 UTF::encode( char32** Out );
</code></pre>

<p>Here it would output the size of the encoded data's array, and create then write the array to the location pointed by <code>**Out</code>. However this is not a really good way since people might want to manage the arrays themselves, such as using custom allocators. So the next idea I brought up was to make it similar to <code>fread</code> at <code>FILE*</code> where it takes the output array and only fills it with data. But for that I need some function to get how big the array has to be.</p>

<pre><code>U32 UTF::encoded_size_UTF8( );
U32 UTF::encoded_size_UTF16( );
U32 UTF::encoded_size_UTF32( );
</code></pre>

<p>This isn't a bright way to do it, and here comes my real problem, cause then it would totally break any compatibility with <code>wchar_t</code> I might had before. I came up with the following two, also not that good ideas:</p>

<pre><code>template&lt;class T&gt; U32 UTF::encoded_size( ) {
    // Here it'd use sizeof( T ) to determine size.
}
</code></pre>

<p>Call would look like: <code>wchar_t* x = new wchar_t[string.encoded_size&lt;wchar_t&gt;( )];</code></p>

<pre><code>// Or have the following overload:
U32 UTF::encoded_size( const char8* target );
U32 UTF::encoded_size( const char16* target );
U32 UTF::encoded_size( const char32* target );
</code></pre>

<p>Here the targeted variable shall be passed so the matching overloaded function can return the size of the specified type. <code>wchar_t* x = new wchar_t[string.encoded_size( x )];</code></p>

<p>So here, at this point I need advises and ideas which with I can keep <code>wchar_t*</code>. I know my explanation is quite bad, and probably not that meaningful, but I just can't really express myself.</p>
