title: Apache Http Client and Load Balancers
body: <p>After spending a few hours reading the Http Client documentation and source code I have decided that I should definitely ask for help here.</p>
  
  <p>I have a load balancer server using a round-robin algorithm somewhat like this</p>
  
  <p><code>
                           +---&gt; RESTServer1 
  client --&gt; load balancer +---&gt; RESTServer2
                           +---&gt; RESTServer3
  </code></p>
  
  <p>Our client is using HttpClient to direct requests to our load balancer server, which in turn round-robins the requests to the corresponding RESTServer.</p>
  
  <p>Now, Apache HttpClient creates, by default, <a href="http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html" rel="noreferrer">a pool of connections</a> (2 per route by default). This connections are by default persistent connections since I am using Http v1.1 and my servers are emitting Connection: Keep-Alive headers.</p>
  
  <p>So, the problems is that since HttpClient creates this persistent connections, then those connections are no longer subject to round-robing algorithm at the balancer level. They always hit the same server every time. </p>
  
  <p>This creates two problems: </p>
  
  <ol>
  <li>I can see that sometimes one or more of the balanced servers are overloaded with traffic, whereas one ore more of the other servers are idle; and </li>
  <li>even if I take one of my REST servers out of the balancer, it stills receives requests while the persistent connections are alive.</li>
  </ol>
  
  <p>Definitely this is not the intended behavior.</p>
  
  <p>I suppose I could force a <code>Connection: close</code> header in my responses, or I could run HttpClient without a connection pool or with a <a href="http://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/ConnectionReuseStrategy.html?is-external=true" rel="noreferrer">NoConnectionReuseStrategy</a>. But <a href="https://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html" rel="noreferrer">the documentation</a> for HttpClient states that the idea behind the use of a pool is to improve performance by avoiding having to open a socket every time and doing all the TPC handshaking and related stuff. So, I have to conclude that the use of a connection pool is beneficial to the performance of my applications.</p>
  
  <p>So my question here, is there a way to use persistent connections with a load-balancer in the way or am I forced to use non-persistent connections for this scenario? </p>
  
  <p>I want the performance that comes with reusing connections, but I want them properly load-balanced. Any thoughts on how I can configure this scenario with Apache Http Client if at all possible?</p>