title: Apache HttpClient 4.x behaving strange when uploading larger files?
body: <p>I'm developing and testing a little straight-forward client-server application using java (and scala). </p>
  
  <p>The <strong>server</strong> is based on <code>com.sun.net.httpserver.HttpServer</code> and allows the upload of files via a basic RESTful interface using POST and PUT operations. The upload operation is restricted using <a href="http://www.ietf.org/rfc/rfc2617.txt" rel="nofollow">Digest authentication</a> which we implemented by ourselves, is tested and works in browsers, curl and <code>Apache HttpClient</code>.</p>
  
  <p>The <strong>upload client</strong> wraps <code>Apache HttpClient 4.1.2</code> and executes PUT operations over http to upload file entities. The content-type of the file is specified as <code>application/xml</code> in the header and only a single file is uploaded at a time.</p>
  
  <p>When uploading files of <strong>different sizes</strong> a strange behaviour could be observed:</p>
  
  <ul>
  <li>Files with sizes less or equals to 1.076.006 Byte are uploaded
  <strong>successfully</strong>.</li>
  <li>Files with sizes greater or equals to 1.122.158 Bytes
  <strong>fail</strong> with a <code>java.net.SocketException: Broken pipe</code>.</li>
  </ul>
  
  <p>(<em>The exact critical size is unknown since I've created files with different sizes manually to approximate the max working size</em>)</p>
  
  <p>The reason for the broken pipe is, that <strong>the client somehow ignored the <code>www-authenticate</code>-response</strong> uploading files of that size, as is documented by the server logs. "Ignore" means, that it just send multiple (4) messages containing no authentication header at all.
  But <em>smaller files</em> work well and the client sends an authentication request with the proper challenge-response correctly immediately after the <code>www-authenticate</code>-response as it should be.</p>
  
  <p><strong>The upload works in curl with files of all sizes, so no problem there.</strong></p>
  
  <p>So at this point, one could say: "There is some bug in your client." Okay, I kind of hope so, but <strong>I've also tried an open-source java <a href="http://code.google.com/p/rest-client/" rel="nofollow">RESTclient</a></strong> (also wrapping apache httpclient) and it has <em>exactly</em> the same behaviour! </p>
  
  <p>We tried it using this client over the internet and its also the same as described. So right now, I just hope I've missed to set something important in <code>Apache HttpClient</code> which leads to this erroneous behaviour and the developer of the open-source RESTclient missed it as well... any ideas what it could be would be great!</p>