title: Best way to have a fast access key-value storage for huge dataset (5 GB)
body: <p>There is a dataset of ~5GB in size. This big dataset just has a key-value pair per line. 
  Now this needs to be read for the value of keys some billion times. </p>
  
  <p>I have already tried disk based approach of <a href="https://github.com/jankotek/MapDB" rel="nofollow">MapDB</a>, but it throws <code>ConcurrentModification Exception</code> and isn't mature enough to be used in production environment yet.</p>
  
  <p>I also don't want to have it in a DB and make the call billion times (Though, certain level of in-memory caching can be done here).</p>
  
  <p>Basically, I need to access these key-value dataset in mapper/reducer of a hadoop's job step.</p>