title: Memory-efficient way to store large List<Map<String,String>> where many map entries are identical
body: <p>I'm looking for a memory-efficient way to store tabular data typically consisting of about 150000 rows x 200 columns.
  The cell values are Strings with lengths somewhere in the range of 0-200 characters.</p>
  
  <p>The data rows are initially generated by taking all possible combinations of rows from smaller tables. So while all rows are unique, the columns contain many copies of the same value.
  The data is not read-only. Some of the columns (typically up to 20 of the 200) get updated with values that depend on the values of other columns. And new columns (also about 20 I'd expect) with computed values are going to be added to the table.</p>
  
  <p>The existing legacy code heavily depends on the data being stored in a <code>List</code> of <code>Map&lt;String, String&gt;</code>s that map column name to cell value.
  But the current implementation, an <code>ArrayList&lt;HashMap&lt;String,String&gt;&gt;</code>, is taking many gigabytes of memory.</p>
  
  <p>I tried calling <code>String.intern()</code> on the keys and values that get inserted into the <code>HashMap</code>. That halved the memory footprint. But it still seems horribly inefficient to keep all those identical <code>Map.Entry</code>s around.</p>
  
  <p>So I was wondering: Can you suggest a more memory-efficient data structure to somehow share the identical column values but that would allow me to keep the external <code>List&lt;Map&lt;String, String&gt;&gt;</code> interface the same?</p>
  
  <p>We already have guava on the class path so using collections from guava is fine.</p>