title: How can I convert surrogate pairs to normal string in Python?
body: <p>This is a follow-up to <a href="https://stackoverflow.com/questions/38106422/converting-to-emoji">Converting to Emoji</a>. In that question, the OP had a <code>json.dumps()</code>-encoded file with an emoji represented as a surrogate pair - <code>\ud83d\ude4f</code>. S/he was having problems reading the file and translating the emoji correctly, and the correct <a href="https://stackoverflow.com/a/38145581/1426065">answer</a> was to <code>json.loads()</code> each line from the file, and the <code>json</code> module would handle the conversion from surrogate pair back to (I'm assuming UTF8-encoded) emoji.</p>
  
  <p>So here is my situation: say I have just a regular Python 3 unicode string with a surrogate pair in it:</p>
  
  <pre><code>emoji = "This is \ud83d\ude4f, an emoji."
  </code></pre>
  
  <p>How do I process this string to get a representation of the <a href="http://apps.timwhitlock.info/unicode/inspect?s=%F0%9F%99%8F" rel="noreferrer">emoji</a> out of it? I'm looking to get something like this:</p>
  
  <pre><code>"This is , an emoji."
  # or
  "This is \U0001f64f, an emoji."
  </code></pre>
  
  <p>I've tried:</p>
  
  <pre><code>print(emoji)
  print(emoji.encode("utf-8")) # also tried "ascii", "utf-16", and "utf-16-le"
  json.loads(emoji) # and `.encode()` with various codecs
  </code></pre>
  
  <p>Generally I get an error similar to <code>UnicodeEncodeError: XXX codec can't encode character '\ud83d' in position 8: surrogates no allowed</code>.</p>
  
  <p>I'm running Python 3.5.1 on Linux, with <code>$LANG</code> set to <code>en_US.UTF-8</code>. I've run these samples both in the Python interpreter on the command line, and within IPython running in Sublime Text - there don't appear to be any differences.</p>