title: JVM Reuse MapReduce 2.0 YARN
body: <p>I have a large number of input splits (about 50.000) created by small files that I want to process with Hadoop. I have, however, only 256 containers available to process it.</p>
  
  <p>The job itself uses lots of CPU but considerable little memory.</p>
  
  <p>I am using Hadoop 2.3 and was taking a look in the JVM reuse feature from MapReduce 1.0</p>
  
  <p>I have also read about uber tasks, but it didn't look like the same thing - or I had a different understand from JVM reuse.</p>
  
  <p>Since I have lots of small files (and am generating a inputSplit per file), I wanted to create a single JVM machine per container and run as many sequential Map tasks as possible per already allocated JVM. This would reduce the JVM allocating time overhead.</p>
  
  <p>I guess that for each input split a new mapper would be allocated and thus a new JVM, am I right?</p>
  
  <p>How can i do such thing in YARN?</p>
  
  <p>Oh, I know that also I can use compressing in order to increase the inputSplit size, but for this exact application this is not viable.</p>
  
  <p>Best Regards,
  Marco Lotz</p>