title: Hadoop: Unable to allocate more than 8 GB Memory to my nodes
body: <p>My hadoop setup was only allocating 8 GB of ememory to each node even tough my machines have 126GB ram and 32 CPU.</p>
  
  <p>I added the following properties to my yarn-site.xml to allocate 24 GB memory to each node.</p>
  
  <pre><code>&lt;property&gt;
     &lt;name&gt;yarn.scheduler.minimum-allocation-mb&lt;/name&gt;
      &lt;value&gt;1024&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
     &lt;name&gt;yarn.scheduler.maximum-allocation-mb&lt;/name&gt;
      &lt;value&gt;2048&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
     &lt;name&gt;yarn.nodemanager.resource.memory-mb&lt;/name&gt;
      &lt;value&gt;24576&lt;/value&gt;
  &lt;/property&gt;
  </code></pre>
  
  <p>I also added following to the mapred-site.xml</p>
  
  <pre><code>&lt;property&gt;
    &lt;name&gt;mapreduce.tasktracker.map.tasks.maximum&lt;/name&gt;
    &lt;value&gt;32&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;mapreduce.tasktracker.reduce.tasks.maximum&lt;/name&gt;
    &lt;value&gt;16&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;mapreduce.job.reduce.slowstart.completedmaps&lt;/name&gt;
    &lt;value&gt;0.95&lt;/value&gt;
  &lt;/property&gt;
  </code></pre>
  
  <p>Now my nodes do show 24 GB each of allocated memory but when I run my task lot of reduce tasks start before the mappers are done and all tasks keep failing. Most of them fail with this exception:</p>
  
  <pre><code>15/04/09 19:24:45 INFO mapreduce.Job: Task Id : attempt_1428621766709_0001_m_000031_2, Status : FAILED
  Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
  org.apache.hadoop.util.Shell$ExitCodeException:
  </code></pre>
  
  <p>The same process used to run earlier with default configuration of 8 GB per node. It would take couple of hours to complete but used to work. Now I cant even get the process to run.</p>
  
  <p>Any pointers here will be really helpful.</p>