title: Curl: get UTF-8 data from site with incorrect charset
body: <p>I scrape some sites that occasionally have UTF-8 characters in the title, but that don't specify UTF-8 as the charset (qq.com is an example).  When I use look at the website in my browser, the data I want to copy (i.e. the title) looks correct (Japanese or Chinese..not too sure).  I can copy the title and paste it into the terminal and it looks exactly the same.  I can even write it to the DB and when I retrieve from the DB it still looks the same, and correct.</p>
  
  <p>However, when I use cURL, the data that gets printed is wrong.  I can run cURL from the command line or use PHP .. when it's printed to the terminal it's clearly incorrect, and it remains that way when I store it to the DB (remember: the terminal <em>can</em> display these characters properly).  I've tried all eligible combinations of the following:</p>
  
  <ul>
  <li>Setting <code>CURLOPT_BINARYTRANSFER</code> to <code>true</code></li>
  <li><code>mb_convert_encoding($html, 'UTF-8')</code></li>
  <li><code>utf8_encode($html)</code></li>
  <li><code>utf8_decode($html)</code></li>
  </ul>
  
  <p>None of these display the characters as expected.  This is very frustrating since I can get the right characters so easily just by visiting the site, but cURL can't.  I've read a lot of suggestions such as this one: <a href="https://stackoverflow.com/questions/4426852/how-to-get-web-page-title-with-curl-in-php-from-web-sites-of-different-charset">How to get web-page-title with CURL in PHP from web-sites of different CHARSET?</a></p>
  
  <p>The solution in general seems to be "convert the data to UTF-8."  To be honest, I don't actually know what that means.  Don't the above functions convert the data to UTF-8?  Why isn't it already UTF-8?  What is it, and why does it display properly in some circumstances, but not for cURL?</p>