title: php's json_encode and character representation
body: <p>I'll try to present it as simple as I can:
  I use json_encode() to encode a number of utf-8 strings from different languages and I notice that characters remain unchanged when they belong to ASCII table but everything else is returned as '\unnnn', where 'nnnn' a hexadecimal number.</p>
  
  <p>See the code:</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"&gt; 
  &lt;html xmlns="http://www.w3.org/1999/xhtml"&gt;
  &lt;head&gt;
  &lt;meta http-equiv="content-type" content="application/xhtml+xml; charset=UTF-8" /&gt;
   &lt;title&gt;Multibyte string functions&lt;/title&gt;
  &lt;/head&gt;
  &lt;body&gt;
  &lt;h3&gt;Multibyte string functions&lt;/h3&gt;
  &lt;p&gt;
  &lt;?php
  //present json encode errors nicely:
  //assign integer values to keys and error names to values
  echo '&lt;br /&gt;&lt;b&gt;Define JSON errors&lt;/b&gt;&lt;br /&gt;';
  $constants = get_defined_constants(true);
  $json_errors = array();
  foreach ($constants["json"] as $name =&gt; $value) {
     if (!strncmp($name, "JSON_ERROR_", 11)) {
        $json_errors[$value] = $name;
     }
  }
  echo nl2br(print_r($json_errors, true), true);
  
  //Display current detection order
  echo "&lt;br /&gt;&lt;b&gt;Current detection order 'mb_detect_order()':&lt;/b&gt; ", implode(", ", mb_detect_order());
  //Display internal encoding
  echo "&lt;br /&gt;&lt;b&gt;Internal encoding 'mb_internal_encoding()':&lt;/b&gt; ",  mb_internal_encoding();
  //Get current language
  echo "&lt;br /&gt;&lt;b&gt;Current detection language 'mb_language()' ('neutral' for utf8):&lt;/b&gt; ", mb_language();
  
  //our test data
  //a nowdoc that can break a &lt;input&gt; field;
  $str = &lt;&lt;&lt;'STR'
  O'Reilly(\n) "&amp; 'Big\Two @ &lt;span&gt;bo\tld&lt;/span&gt;"
  STR;
  $strings = array(
     $str,
     "Latin: tell me the answer and I might find the question!",
     "Greek: πες μου την ερώτηση και ίσως βρω την απάντηση!",
     "Chinese simplified: 告诉我答复，并且我也许发现问题!",
     "Arabic: أخبرني الاجابة, انا قد تجد مسالة!",
     "Portuguese: mais coisas a pensar sobre diário ou dois!",
     "French: plus de choses à penser à journalier ou à deux!",
     "Spanish: ¡más cosas a pensar en diario o dos!",
     "Italian: più cose da pensare circa giornaliere o due!",
     "Danish: flere ting å tenke på hver dag eller to!",
     "Chech: Další věcí, přemýšlet o každý den nebo dva!",
     "German: mehr über Spaß spät schönen",
     "Albanian: më vonë gjatë fun bukur",
     "Hungarian: több mint szórakozás késő csodálatos kenyér"
  );
  
  //show encoding and then encode
  foreach( $strings as $string ){
     echo "&lt;br /&gt;&lt;br /&gt;$string :", mb_detect_encoding($string);
     $json = json_encode($string);
     echo "&lt;br /&gt;Error? ", $json_errors[json_last_error()];
     echo '&lt;br /&gt;json=', $json;
  }
  </code></pre>
  
  <p>The above code will output:</p>
  
  <pre><code>Define JSON errors
  Array
  (
  [0] =&gt; JSON_ERROR_NONE
  [1] =&gt; JSON_ERROR_DEPTH
  [2] =&gt; JSON_ERROR_STATE_MISMATCH
  [3] =&gt; JSON_ERROR_CTRL_CHAR
  [4] =&gt; JSON_ERROR_SYNTAX
  [5] =&gt; JSON_ERROR_UTF8
  )
  
  Current detection order 'mb_detect_order()': ASCII, UTF-8
  Internal encoding 'mb_internal_encoding()': ISO-8859-1
  Current detection language 'mb_language()' ('neutral' for utf8): neutral
  
  O'Reilly(\n) "&amp; 'Big\Two @ bo\tld" :ASCII
  Error? JSON_ERROR_NONE
  json="O'Reilly(\\n) \"&amp; 'Big\\Two @ bo\\tld&lt;\/span&gt;\""
  
  Latin: tell me the answer and I might find the question! :ASCII
  Error? JSON_ERROR_NONE
  json="Latin: tell me the answer and I might find the question!"
  
  Greek: πες μου την ερώτηση και ίσως βρω την απάντηση! :UTF-8
  Error? JSON_ERROR_NONE
  json="Greek: \u03c0\u03b5\u03c2 \u03bc\u03bf\u03c5 \u03c4\u03b7\u03bd \u03b5\u03c1\u03ce\u03c4\u03b7\u03c3\u03b7 \u03ba\u03b1\u03b9 \u03af\u03c3\u03c9\u03c2 \u03b2\u03c1\u03c9 \u03c4\u03b7\u03bd \u03b1\u03c0\u03ac\u03bd\u03c4\u03b7\u03c3\u03b7!"
  
  Chinese simplified: 告诉我答复，并且我也许发现问题! :UTF-8
  Error? JSON_ERROR_NONE
  json="Chinese simplified: \u544a\u8bc9\u6211\u7b54\u590d\uff0c\u5e76\u4e14\u6211\u4e5f\u8bb8\u53d1\u73b0\u95ee\u9898!"
  
  Arabic: أخبرني الاجابة, انا قد تجد مسالة! :UTF-8
  Error? JSON_ERROR_NONE
  json="Arabic: \u0623\u062e\u0628\u0631\u0646\u064a \u0627\u0644\u0627\u062c\u0627\u0628\u0629, \u0627\u0646\u0627 \u0642\u062f \u062a\u062c\u062f \u0645\u0633\u0627\u0644\u0629!"
  
  Portuguese: mais coisas a pensar sobre diário ou dois! :UTF-8
  Error? JSON_ERROR_NONE
  json="Portuguese: mais coisas a pensar sobre di\u00e1rio ou dois!"
  
  French: plus de choses à penser à journalier ou à deux! :UTF-8
  Error? JSON_ERROR_NONE
  json="French: plus de choses \u00e0 penser \u00e0 journalier ou \u00e0 deux!"
  
  Spanish: ¡más cosas a pensar en diario o dos! :UTF-8
  Error? JSON_ERROR_NONE
  json="Spanish: \u00a1m\u00e1s cosas a pensar en diario o dos!"
  
  Italian: più cose da pensare circa giornaliere o due! :UTF-8
  Error? JSON_ERROR_NONE
  json="Italian: pi\u00f9 cose da pensare circa giornaliere o due!"
  
  Danish: flere ting å tenke på hver dag eller to! :UTF-8
  Error? JSON_ERROR_NONE
  json="Danish: flere ting \u00e5 tenke p\u00e5 hver dag eller to!"
  
  Chech: Další věcí, přemýšlet o každý den nebo dva! :UTF-8
  Error? JSON_ERROR_NONE
  json="Chech: Dal\u0161\u00ed v\u011bc\u00ed, p\u0159em\u00fd\u0161let o ka\u017ed\u00fd den nebo dva!"
  
  German: mehr über Spaß spät schönen :UTF-8
  Error? JSON_ERROR_NONE
  json="German: mehr \u00fcber Spa\u00df sp\u00e4t sch\u00f6nen"
  
  Albanian: më vonë gjatë fun bukur :UTF-8
  Error? JSON_ERROR_NONE
  json="Albanian: m\u00eb von\u00eb gjat\u00eb fun bukur"
  
  Hungarian: több mint szórakozás késő csodálatos kenyér :UTF-8
  Error? JSON_ERROR_NONE
  json="Hungarian: t\u00f6bb mint sz\u00f3rakoz\u00e1s k\u00e9s\u0151 csod\u00e1latos keny\u00e9r"
  </code></pre>
  
  <p>As you can see in most languages-except English-there is a hexadecimal conversion of utf-8 characters.
  Is it possible to encode by not replacing my unicode characters? Is it safe? What other people do?</p>
  
  <p>You should consider such encodings that are coming from user input in pages and stored to mysql.</p>
  
  <p>Thanks.</p>