title: How to resolve Java heap space with htmlunit
body: <p>We have a multi-threaded application that is used to crawl data from various websites. We use htmlunit to download the content for some of the websites. 
  The issue of Out of memory error has suddenly started to come. Our application uses 30 threads in the deployment server.</p>
  
  <p>We are using following in our deployments:</p>
  
  <pre><code>htmlunit-2.12.jar
  htmlunit-core-js-2.12.jar
  Java: Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
  </code></pre>
  
  <p>For tomcat we use following parameters:</p>
  
  <pre><code>-Xms1024m -Xmx3584m 
  -XX:+UseConcMarkSweepGC
  -XX:PermSize=256m -XX:MaxPermSize=256m
  </code></pre>
  
  <p>Running on:
  CentOS release 6.3 (4GB RAM)</p>
  
  <p>From various suggestion on Java heap memory related to htmlunit, I have tried:</p>
  
  <ol>
  <li>Changing from Java(TM) SE Runtime Environment to OpenJDK Runtime Environment.</li>
  <li>Used webClient.closeAllWindows(); where ever required.</li>
  </ol>
  
  <p>Have found no luck with these changes. The application uses entire heap in under 10 minutes.
  How can we overcome this Out of Memory error issue ?</p>
  
  <p><strong>Edit:</strong> After updating to htmlunit-2.16.jar we get improvement in garbage collection. (along with OpenJDK)</p>
  
  <p><a href="https://i.stack.imgur.com/HuIHt.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/HuIHt.png" alt="enter image description here"></a></p>
  
  <p>Before:(With htmlunit-2.12.jar and Java(TM) SE Runtime Environment (build 1.7.0_79-b15))
  <a href="https://i.stack.imgur.com/xmQq7.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/xmQq7.png" alt="enter image description here"></a></p>
  
  <p>Is version of htmlunit causing problem ?</p>
  
  <p><strong>Edit</strong>
  In the legacy code of the application, webClient were not being closed in some particular case, which caused webClient objects to be created and hence heap size increased. But even after closing the webclient and making it null, the heap space issue persists.</p>
  
  <p>How to resolve this memory heap issue?</p>