title: OutOfMemory when reading big amounts of data using hibernate
body: <p>I need to export big amount of data from database. Here is classes that represents my data:</p>
  
  <pre><code>public class Product{
  ...
  
      @OneToMany
      @JoinColumn(name = "product_id")
      @Cascade({SAVE_UPDATE, DELETE_ORPHAN})
      List&lt;ProductHtmlSource&gt; htmlSources = new ArrayList&lt;ProductHtmlSource&gt;();
  </code></pre>
  
  <p>...
     }</p>
  
  <p><code>ProductHtmlSource</code> - contains big string inside which I actually need to export. </p>
  
  <p>Since size of exported data is bigger than JVM memory I'm reading my data by chunks. Like this:</p>
  
  <pre><code>final int batchSize = 1000;      
  for (int i = 0; i &lt; 50; i++) {
    ScrollableResults iterator = getProductIterator(batchSize * i, batchSize * (i + 1));
    while (iterator.getScrollableResults().next()) {
       Product product = (Product) iterator.getScrollableResults().get(0); 
       List&lt;String&gt; htmls = product.getHtmlSources();
       &lt;some processing&gt;
    }
  </code></pre>
  
  <p>} </p>
  
  <p>Code of <code>getProductIterator</code> :</p>
  
  <pre><code>public ScrollableResults getProductIterator(int offset, int limit) {
          Session session = getSession(true);
          session.setCacheMode(CacheMode.IGNORE);
          ScrollableResults iterator = session
                  .createCriteria(Product.class)
                  .add(Restrictions.eq("status", Product.Status.DONE))
                  .setFirstResult(offset)
                  .setMaxResults(limit)
                  .scroll(ScrollMode.FORWARD_ONLY);
          session.flush();
          session.clear();
  
          return iterator;
      }
  </code></pre>
  
  <p>The problem is that in spite of I clearing session after reading of each data chunk <code>Product</code> objects accumulates somewhere and I'm get OutOfMemory exception. The problem is not in processing block of code even without it I get memory error. The size of batch also is not a problem since 1000 objects easily sit into memory. </p>
  
  <p>Profiler showed that objects accumulates in <code>org.hibernate.engine.StatefulPersistenceContext</code> class. </p>
  
  <p>The stacktrace:</p>
  
  <pre><code>Caused by: java.lang.OutOfMemoryError: Java heap space
      at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:99)
      at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:518)
      at java.lang.StringBuffer.append(StringBuffer.java:307)
      at org.hibernate.type.TextType.get(TextType.java:41)
      at org.hibernate.type.NullableType.nullSafeGet(NullableType.java:163)
      at org.hibernate.type.NullableType.nullSafeGet(NullableType.java:154)
      at org.hibernate.type.AbstractType.hydrate(AbstractType.java:81)
      at org.hibernate.persister.entity.AbstractEntityPersister.hydrate(AbstractEntityPersister.java:2101)
      at org.hibernate.loader.Loader.loadFromResultSet(Loader.java:1380)
      at org.hibernate.loader.Loader.instanceNotYetLoaded(Loader.java:1308)
      at org.hibernate.loader.Loader.getRow(Loader.java:1206)
      at org.hibernate.loader.Loader.getRowFromResultSet(Loader.java:580)
      at org.hibernate.loader.Loader.doQuery(Loader.java:701)
      at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:236)
      at org.hibernate.loader.Loader.loadCollection(Loader.java:1994)
      at org.hibernate.loader.collection.CollectionLoader.initialize(CollectionLoader.java:36)
      at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:565)
      at org.hibernate.event.def.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:63)
      at org.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:1716)
      at org.hibernate.collection.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:344)
      at org.hibernate.collection.AbstractPersistentCollection.read(AbstractPersistentCollection.java:86)
      at org.hibernate.collection.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:109)
      at org.hibernate.collection.PersistentBag.size(PersistentBag.java:225)
      **at com.rivalwatch.plum.model.Product.getHtmlSource(Product.java:76)
      at com.rivalwatch.plum.model.Product.getHtmlSourceText(Product.java:80)
      at com.rivalwatch.plum.readers.AbstractDataReader.getData(AbstractDataReader.java:64)**
  </code></pre>