title: UTF8, codepoints, and their representation in Erlang and Elixir
body: <p>going through Elixir's handling of unicode:</p>
  
  <pre><code>iex&gt; String.codepoints("abc§")
  ["a", "b", "c", "§"]
  </code></pre>
  
  <p>very good, and byte_size/2 of this is not 4 but 5, because the last char is taking 2 bytes, I get that.</p>
  
  <p>The ? operator (or is it a macro? can't find the answer) tells me that</p>
  
  <pre><code>iex(69)&gt; ?§
  167
  </code></pre>
  
  <p>Great; so then I look into the UTF-8 encoding table, and see value <strong>c2 a7</strong> as hex encoding for the char. That means the two bytes (as witnessed by byte_size/1) are c2 (94 in decimal) and a7 (167 in decimal). That 167 is the result I got when evaluating <strong>?§</strong> earlier. What I don't understand, exactly, is.. why that number is a "code point", as per the description of the <strong>?</strong> operator. When I try to work backwards, and evaluate the binary, I get what I want:</p>
  
  <pre><code>iex(72)&gt; &lt;&lt;0xc2, 0xa7&gt;&gt;
  "§"
  </code></pre>
  
  <p>And to make me go completely bananas, this is what I get in Erlang shell:</p>
  
  <pre><code>24&gt; &lt;&lt;167&gt;&gt;.
  &lt;&lt;"§"&gt;&gt;
  25&gt; &lt;&lt;"\x{a7}"&gt;&gt;.
  &lt;&lt;"§"&gt;&gt;
  26&gt; &lt;&lt;"\x{c2}\x{a7}"&gt;&gt;.
  &lt;&lt;"§"/utf8&gt;&gt;
  27&gt; &lt;&lt;"\x{c2a7}"&gt;&gt;.    
  &lt;&lt;"§"&gt;&gt;
  </code></pre>
  
  <p>!! while Elixir is only happy with the code above... what is it that I don't understand? Why is Erlang perfectly happy with a single byte, given that Elixir insists that char takes 2 bytes - and Unicode table seems to agree?</p>