title: How can I ensure that my HttpClient 4.1 does not leak sockets?
body: <p>My server uses data from an internal web service to construct its response, on a per request basis.  I'm using Apache HttpClient 4.1 to make the requests.  Each initial request will result in about 30 requests to the web service.  Of these, 4 - 8 will end up with sockets stuck in CLOSE_WAIT, which never get released.  Eventually these stuck sockets exceed my ulimit and my process runs out of file descriptors.</p>
  
  <p>I don't want to just raise my ulimit (1024), because that will just mask the problem.</p>
  
  <p>The reason I've moved to HttpClient is that java.net.HttpUrlConnection was behaving the same way.</p>
  
  <p>I have tried moving to a SingleClientConnManager per request, and calling client.getConnectionManager().shutdown() on it, but sockets still end up stuck.</p>
  
  <p>Should I be trying to solve this so that I end up with 0 open sockets while there are no running requests, or should I be concentrating on request persistence and pooling?</p>
  
  <p>For clarity I'm including some details which may be relevant:</p>
  
  <p>OS: Ubuntu 10.10</p>
  
  <p>JRE: 1.6.0_22</p>
  
  <p>Language: Scala 2.8</p>
  
  <p>Sample code:</p>
  
  <pre><code>val cleaner = Executors.newScheduledThreadPool(1) 
  private val client = {
      val ssl_ctx = SSLContext.getInstance("TLS")
      val managers = Array[TrustManager](TrustingTrustManager)
      ssl_ctx.init(null, managers, new java.security.SecureRandom())
      val sslSf = new org.apache.http.conn.ssl.SSLSocketFactory(ssl_ctx, SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER)
      val schemeRegistry = new SchemeRegistry()
      schemeRegistry.register(new Scheme("https", 443, sslSf))
      val connection = new ThreadSafeClientConnManager(schemeRegistry)
      object clean extends Runnable{ 
          override def run = {
              connection.closeExpiredConnections
              connection.closeIdleConnections(30, SECONDS)
          }
      }
      cleaner.scheduleAtFixedRate(clean,10,10,SECONDS)
      val httpClient = new DefaultHttpClient(connection)
      httpClient.getCredentialsProvider().setCredentials(new AuthScope(AuthScope.ANY), new UsernamePasswordCredentials(username,password))
      httpClient
  }
  val get = new HttpGet(uri)
  val entity = client.execute(get).getEntity
  val stream = entity.getContent
  val justForTheExample = IOUtils.toString(stream)
  stream.close()
  </code></pre>
  
  <p>Test: netstat -a | grep {myInternalWebServiceName} | grep CLOSE_WAIT</p>
  
  <p>(Lists sockets for my process that are in CLOSE_WAIT state)</p>
  
  <p>Post comment discussion:</p>
  
  <p>This code now demonstrates correct usage.</p>