title: Simple UTF8->UTF16 string conversion with iconv
body: <p>I want to write a function to convert a UTF8 string to UTF16 (little-endian). The problem is, the <code>iconv</code> function does not seem to let you know in advance how many bytes you'll need to store the output string.</p>
  
  <p>My solution is to start by allocating <code>2*strlen(utf8)</code>, and then run <code>iconv</code> in a loop,  increasing the size of that buffer with <code>realloc</code> if necessary:</p>
  
  <pre><code>static int utf8_to_utf16le(char *utf8, char **utf16, int *utf16_len)
  {
      iconv_t cd;
      char *inbuf, *outbuf;
      size_t inbytesleft, outbytesleft, nchars, utf16_buf_len;
  
      cd = iconv_open("UTF16LE", "UTF8");
      if (cd == (iconv_t)-1) {
          printf("!%s: iconv_open failed: %d\n", __func__, errno);
          return -1;
      }
  
      inbytesleft = strlen(utf8);
      if (inbytesleft == 0) {
          printf("!%s: empty string\n", __func__);
          iconv_close(cd);
          return -1;
      }
      inbuf = utf8;
      utf16_buf_len = 2 * inbytesleft;            // sufficient in many cases, i.e. if the input string is ASCII
      *utf16 = malloc(utf16_buf_len);
      if (!*utf16) {
          printf("!%s: malloc failed\n", __func__);
          iconv_close(cd);
          return -1;
      }
      outbytesleft = utf16_buf_len;
      outbuf = *utf16;
  
      nchars = iconv(cd, &amp;inbuf, &amp;inbytesleft, &amp;outbuf, &amp;outbytesleft);
      while (nchars == (size_t)-1 &amp;&amp; errno == E2BIG) {
          char *ptr;
          size_t increase = 10;                   // increase length a bit
          size_t len;
          utf16_buf_len += increase;
          outbytesleft += increase;
          ptr = realloc(*utf16, utf16_buf_len);
          if (!ptr) {
              printf("!%s: realloc failed\n", __func__);
              free(*utf16);
              iconv_close(cd);
              return -1;
          }
          len = outbuf - *utf16;
          *utf16 = ptr;
          outbuf = *utf16 + len;
          nchars = iconv(cd, &amp;inbuf, &amp;inbytesleft, &amp;outbuf, &amp;outbytesleft);
      }
      if (nchars == (size_t)-1) {
          printf("!%s: iconv failed: %d\n", __func__, errno);
          free(*utf16);
          iconv_close(cd);
          return -1;
      }
  
      iconv_close(cd);
      *utf16_len = utf16_buf_len - outbytesleft;
  
      return 0;
  }
  </code></pre>
  
  <p>Is this really the best way to do it? Repeated <code>realloc</code>s seems wasteful, but without knowing what character sequences could be in the utf8, and what they would result in in utf16, I don't know if I can make a better guess for the initial buffer size than <code>2*strlen(utf8)</code>.</p>