title: C++ iterate utf-8 string with mixed length of characters
body: <p>I need to loop over a utf-8 string and get each character of the string. There might be different types of characters in the string, e.g. numbers with the length of one byte, Chinese characters with the length of three bytes, etc. </p>
  
  <p>I looked at this <a href="https://stackoverflow.com/questions/2852895/c-iterate-or-split-utf-8-string-into-array-of-symbols#2856241">post</a> and it can do 80% of the job, except that when the string has 3-byte chinese characters before 1-byte numbers, it will see the numbers also as having 3 bytes and print the numbers as 1** where * is gibberish. </p>
  
  <p>To give an example, if the string is '今天周五123', the result will be:</p>
  
  <p>今<br>
  天<br>
  周<br>
  五<br>
  1**<br>
  2**<br>
  3**  </p>
  
  <p>where * is gibberish. However if the string is '123今天周五', the numbers will print out fine. </p>
  
  <p>The minimally adapted code from the above mentioned <a href="https://stackoverflow.com/questions/2852895/c-iterate-or-split-utf-8-string-into-array-of-symbols#2856241">post</a> is copied here:</p>
  
  <pre><code>#include &lt;iostream&gt;
  #include "utf8.h"
  
  using namespace std;
  
  int main() {    
      string text = "今天周五123";
  
      char* str = (char*)text.c_str();    // utf-8 string
      char* str_i = str;                  // string iterator
      char* end = str+strlen(str)+1;      // end iterator
  
      unsigned char symbol[5] = {0,0,0,0,0};
  
      cout &lt;&lt; symbol &lt;&lt; endl;
  
      do
      {
          uint32_t code = utf8::next(str_i, end); // get 32 bit code of a utf-8 symbol
          if (code == 0)
              continue;
  
          cout &lt;&lt; "utf 32 code:" &lt;&lt; code &lt;&lt; endl;
  
          utf8::append(code, symbol); // initialize array `symbol`
  
          cout &lt;&lt; symbol &lt;&lt; endl;
  
      }
      while ( str_i &lt; end );
  
      return 0;
  }
  </code></pre>
  
  <p>Can anyone help me here? I am new to c++ and although I checked the documentation of <a href="http://utfcpp.sourceforge.net/#introduction" rel="noreferrer">utf8 cpp</a>, I still have no idea where the problem is. I think the library was created to handle such issues where you have utf-8 encodings with different lengths, so there should be a way to do this... Have been struggling with this for two days...</p>