   C++ 0x strings
  I read that char has been modified to be both at least the size necessary to store an eight-bit coding of UTF-8 and large enough to contain any member of the compiler's basic execution character set. (from  wikipedia).1) Does this mean the size of char is going to change? As far as I know, UTF-8 has more characters than ASCII.The following shows how to create string literals for each of these encodings:u8"I'm a UTF-8 string."u"This is a UTF-16 string."U"This is a UTF-32 string."2) Ok I tried this Visual Studio 2010 and it doesn't work. Does anyone know a compiler that has implemented this?3) What about string objects, are there going to be std::string8, std::string16 etc? If nothing of these happens, how are we going to store our Unicode strings? (I hope not in character arrays)   
Quote:Original post by sheep19I read that char has been modified to be both at least the size necessary to store an eight-bit coding of UTF-8 and large enough to contain any member of the compiler's basic execution character set. (from  wikipedia).1) Does this mean the size of char is going to change? As far as I know, UTF-8 has more characters than ASCII.UTF-8 is always 8 bits, hence the name. So nothing really new there :)Quote:2) Ok I tried this Visual Studio 2010 and it doesn't work. Does anyone know a compiler that has implemented this?I think this is the most up-to-date summary. For more C++0x features, I'd recommend gcc: C++0x Core Language Features.Quote:3) What about string objects, are there going to be std::string8, std::string16 etc? If nothing of these happens, how are we going to store our Unicode strings? (I hope not in character arrays)According to 21.3 String classes of the holy upcoming standard (refering to n3000.pdf, but a more recent draft is already available), there will be string, u16string, u32string, wstring, all of which are specializations of basic_string<>.;Quote:Original post by sheep19I read that char has been modified to be both at least the size necessary to store an eight-bit coding of UTF-8 and large enough to contain any member of the compiler's basic execution character set. (from  wikipedia).The definition of char type has not changed.  It is large enough to contain any member of the compiler's basic execution set [3.9.1](1).The meaning of character array string initializers has changed to encompass various Unicode literals [8.5.2].  String literals and char types are not the same thing, although you use string literals to initialize arrays of char types, so they are related.  The standard assigns no semantic meaning to the data stored in an array of a char type.Quote:2) Ok I tried this Visual Studio 2010 and it doesn't work. Does anyone know a compiler that has implemented this?If you pull GCC from head and rebuild it yourself you should get support for the various string literals, but I don't think support for raw strings is there yet (I could be wrong -- I haven't pulled recently).Quote:3) What about string objects, are there going to be std::string8, std::string16 etc? If nothing of these happens, how are we going to store our Unicode strings? (I hope not in character arrays)From the draft standard.// basic_string typedef namestypedef basic_string<char> string;typedef basic_string<char16_t> u16string;typedef basic_string<char32_t> u32string;typedef basic_string<wchar_t> wstring;An interesting observation is that the standard provides no such explicit typedefs for regexes of type char16_t or char32_t.  I believe this may be an oversight that will be corrected before the final release.   ;  Thanks for the answers. Well it seems odd to me that they hadn't added UTF-8 support a long time ago :/Another question.when using a wstring:wstring wstr = L"αβγδεζ";wcout << str;1)This prints a number, not the string. What is char16_t and char32_t? Are they real types or are they typedefs of int? -> Printing those will print the correct unicode characters or just numbers?2)I'm pretty sure that string str = "αβγδεζ";cout << str;will print correctly right?   ;  When I was looking at the include files of the windows API, most of the weird types like __in64 were defined as common types with typedefs. My guess would be that things such as char16_t would simply be defined as a 2 byte short so:typedef unsigned short char16_t;In C this is mostly done with defines so instead we got:#define char16_t unsigned shortBut more likely then not, this is probably a typedef. Defines have names less awkward like WORD or BOOL or UINT. I also think that the _t might stand for typedef so idk >.> I'm not an expert on these.For 32 bits we use a 4 byte variable which is usually a long. Of course there would be slight complications that would concern the use of that kind of variables. Mainly, they are integer types so special functions would have to be required to retrieve character values within those. On the other hand, there can always be a built in type into the compiler; however, the built in types probably won't have names such as char16_t. If I understand the naming convention well, it will be something like:extended char;or we'll flip around and use:short char; - 8 bitschar;  - 16 bitslong char;  - 32 bitsand yeah, that's about it :)   ;Quote:Original post by boogyman19946... My guess would be that things such as char16_t would simply be defined as a 2 byte short so:typedef unsigned short char16_t; I have not looked at this part of the draft yet I highly doubt it for a number of reasons. Firstly as you point out later in your post this would cause major problems, secondly short's size is not specified and lastly would it be unsigned, signed or implementation specfic?Quote:But more likely then not, this is probably a typedef. Again I doubt it, more like it will be a distinct type.Quote:I also think that the _t might stand for typedef _t suffix means type most of the time.Quote:For 32 bits we use a 4 byte variable which is usually a long. C++0x has sized types, but note it is the bit size you want to check not byte size as a C++ byte can be bigger or smaller than 8 bits.   ;Quote:Original post by sheep19Thanks for the answers. Well it seems odd to me that they hadn't added UTF-8 support a long time agoUnicode was still in near-constant flux when the C++ standard was proclaimed.  It was also not in widespread use.Quote:when using a wstring:wstring wstr = L"αβγδεζ";wcout << str;That's going to depend on what your local locale is.  Given that the semantiocs of a wide-character string are not defined by the language and that your operating environment may have a different idea of what is represent, and that everything is platform-specific, who know what is expected when in that line of code?Quote:What is char16_t and char32_t? Are they real types or are they typedefs of int? -> Printing those will print the correct unicode characters or just numbers?[3.9.1](5)  Type wchar_t is a distinct type whose values can represent distinct codes for all members of the largest  extended character set specified among the supported locales (22.3.1). Type wchar_t shall have the same  size, signedness, and alignment requirements (3.11) as one of the other integral types, called its underlying  type. Types char16_t and char32_t denote distinct types with the same size, signedness, and alignment as  uint_least16_t and uint_least32_t, respectively, in <stdint.h>, called the underlying types.They're distinct types to avoid overload resolution problems.  They are not typedefs.Quote:I'm pretty sure that string str = "αβγδεζ";cout << str;will print correctly right?Heh.  Unlikely, depending on your definition of "correct."  If the character set of your editor the same as that of your runtime and whaever you're using to view the output?;Quote:Original post by CmpDevQuote:Original post by boogyman19946... My guess would be that things such as char16_t would simply be defined as a 2 byte short so:typedef unsigned short char16_t; I have not looked at this part of the draft yet I highly doubt it for a number of reasons. Firstly as you point out later in your post this would cause major problems, secondly short's size is not specified and lastly would it be unsigned, signed or implementation specfic?I don't doubt it at all. It would cause problems indeed. And I would guess it to be unsigned due to the keyword "unsigned" in front of "short".Quote:Original post by CmpDevQuote:But more likely then not, this is probably a typedef. Again I doubt it, more like it will be a distinct type.When it's implemented, I don't doubt it will be.Quote:Original post by CmpDevQuote:I also think that the _t might stand for typedef _t suffix means type most of the time.I see :D Learn something everyday.Quote:Original post by CmpDevQuote:For 32 bits we use a 4 byte variable which is usually a long. C++0x has sized types, but note it is the bit size you want to check not byte size as a C++ byte can be bigger or smaller than 8 bits.Hmm, that I wouldn't know. Since I started learning about memory block sizes, I figured all bytes were created equal, so 8 bits = byte.;Quote:Original post by boogyman19946Quote:Original post by CmpDevQuote:For 32 bits we use a 4 byte variable which is usually a long. C++0x has sized types, but note it is the bit size you want to check not byte size as a C++ byte can be bigger or smaller than 8 bits.Hmm, that I wouldn't know. Since I started learning about memory block sizes, I figured all bytes were created equal, so 8 bits = byte.Nope...sizeof is defined as returning the number of bytes in a the expression passed to it, and also is defined such that sizeof(char) == 1. char is defined as the smallest addressable unit. Usually that means a byte, but some systems dont defined bytes to be 8 bits, or the like. An example is old networking telnet systems which used 7 bits with a parity bit for error control.   ;Quote:Original post by CmpDeva C++ byte can be bigger or smaller than 8 bits.Actually, the minimum size for a C++ byte is 8 bits. It can be larger, but it can't be smaller.