title: Webpage charcode extraction on App Engine with Async UrlFetch
body: <p>There ave been many discussions about determining the character encoding of text files, and webpages.</p>
  
  <p>For webpages it seems the best/simple way is to use a library which takes a Url as the input and returns the properly encoded string. Since the library is fetching the document it can use the HTTP Header to help determine the encoding.</p>
  
  <ol>
  <li><a href="http://htmlcleaner.sourceforge.net/download" rel="nofollow noreferrer">http://htmlcleaner.sourceforge.net/download</a>.</li>
  <li><a href="https://stackoverflow.com/questions/238547/how-do-you-programmatically-download-a-webpage-in-java">How do you Programmatically Download a Webpage in Java</a></li>
  <li><a href="https://stackoverflow.com/questions/5077630/character-encoding-in-a-web-page-using-java">character encoding in a web page using java</a> </li>
  <li><a href="http://docs.oracle.com/javase/6/docs/technotes/guides/intl/encoding.doc.html" rel="nofollow noreferrer">http://docs.oracle.com/javase/6/docs/technotes/guides/intl/encoding.doc.html</a></li>
  </ol>
  
  <p>If we want to fetch documents using the UrlFetch, specifically the async api, whats the best approach, library? for determining the encoding.</p>
  
  <p>Are there any libraries that integrate (or could easily modified to integrate) with async urlfetch?</p>