title: Jackson->Jackson + HttpPost = "Invalid UTF-8 middle byte", Setting Mime and Encoding
body: <p>I'm using the Apache HTTP Client libs and Jackson in my client.  When I post JSON to the server I get the error:</p>
  
  <pre><code>org.codehaus.jackson.JsonParseException: Invalid UTF-8 middle byte 0x65
   at [Source: HttpInputOverHTTP@22a4ac95; line: 1, column: 81]
  </code></pre>
  
  <p>If I don't set any headers than I get <code>invalid media type</code>, which makes sense.</p>
  
  <p>If I use curl and the same headers, the server accepts it, so I think the server is OK (and just a coincidence that it's also using Jackson)</p>
  
  <p>These is the document; I've hard coded it as a Java literal using only 8 bit characters to avoid any other place for mangling to happen</p>
  
  <pre><code>// "Stra\u00DFe" = "Straße"
  static String TINY_UTF8_DOC = "[{ \"id\" : \"2\", \"fields\" : { \"subject\" : [{ \"name\" : \"subject\", \"value\" : \"Stra\u00DFe\" }] } }]";
  </code></pre>
  
  <p>Here's the code I've been using, and comments with the various attempts:</p>
  
  <pre><code>HttpClient httpClient = new DefaultHttpClient();
  HttpPost post = new HttpPost( url );
  
  // Attempt A
  // post.setEntity(  new StringEntity( content )  );
  
  // Attempt B
  // post.setEntity(  new StringEntity( content )  );
  // post.setHeader("Content-Type", "application/json; charset=utf-8");
  
  // Attempt C
  // post.setEntity(  new StringEntity( content, ContentType.create("application/json") )  );
  
  // Attempt D
  // post.setEntity(  new StringEntity( content, ContentType.create("application/json; charset=UTF-8") )  );
  
  // Attempt F
  // post.setEntity(  new StringEntity( content, ContentType.create("application/json; charset=utf-8") )  );
  
  // Attempt G
  // StringEntity params = new StringEntity( content );
  // params.setContentType("application/json; charset=UTF-8");
  // post.setEntity(params);
  
  // And then send to server
  HttpResponse response = httpClient.execute( post );
  int code = response.getStatusLine().getStatusCode();
  // ...etc...
  </code></pre>
  
  <p>Other weird things I've noticed:</p>
  
  <ul>
  <li>For a while this behaved differently on Eclipse on the Mac vs. running a .jar on Linux; clearly that's a symptom of platform-specific encoding or decoding, but I don't know where.  Ironically that <em>broke</em> when I set Eclipse to treat code as UTF-8 (vs. ASCII)  I suspect this is an important clue, but not sure where it fits.</li>
  <li>I've seen times when instead of 2 bytes there's 4 bytes in the stream, though this might have been a different encoding problem when writing to disk, though I was specifically setting UTF-8 on file IO</li>
  <li>When I look at the string entity in the debugger, I see the bytes, but the 8-bit character is a negative number.  When you run through the Two's Compliment math, it is still the correct Unicode code point, so nominally OK, assuming httpclient isn't buggy.</li>
  </ul>
  
  <p>Really out of ideas, and as I said, it works with curl, so I think the server is OK.</p>
  
  <p><strong>Edit:</strong></p>
  
  <p>curl works when posting to the server, but I can't share the server code.  It was pointed out that since curl isn't written in Java, and so perhaps it behaves differently, and therefore the server code could still be suspect.</p>
  
  <p>So as a further test, the code below does <strong>NOT</strong> use the Apache httpclient library, and DOES work when posting to the server.  This proves that the server is fine and there's still something wrong with how I'm using the Apache library on the client side (or maybe it's buggy).</p>
  
  <p><strong>Non-apache-httpclient code, which does work:</strong></p>
  
  <pre><code>import java.io.OutputStream;
  import java.io.OutputStreamWriter;
  import java.net.HttpURLConnection;
  import java.net.URL;
  
  class PostUtf8 {
      static String POST_URL = "http://...";
  
      // \u00DF = LATIN SMALL LETTER SHARP S, looks like letter B
      static String TINY_UTF8_DOC = "[{ \"id\" : \"2\", \"fields\" : { \"subject\" : [{ \"name\" : \"subject\", \"value\" : \"Stra\u00DFe\" }] } }]";
  
      public static void main( String [] args ) throws Exception {
          System.out.println( "Posting to " + POST_URL );
          URL url = new URL( POST_URL );
          HttpURLConnection conn = (HttpURLConnection) url.openConnection();
          conn.setRequestProperty( "Content-Type", "application/json; charset=UTF-8" );
          conn.setRequestMethod("POST");
          conn.setDoOutput(true);
          OutputStream sout = conn.getOutputStream();
          OutputStreamWriter wout = new OutputStreamWriter(sout, "UTF-8" );
          wout.write( TINY_UTF8_DOC );
          wout.flush();
          int result = conn.getResponseCode();
          System.out.println( "Result = " + result );
      }
  }
  </code></pre>