title: Hadoop YARN job is getting stucked at map 0% and reduce 0%
body: <p>I am trying to run a very simple job to test my hadoop setup so I tried with Word Count Example , which get stuck in 0% , so i tried some other simple jobs and each one of them stuck </p>
  
  <pre><code>52191_0003/
  14/07/14 23:55:51 INFO mapreduce.Job: Running job: job_1405376352191_0003
  14/07/14 23:55:57 INFO mapreduce.Job: Job job_1405376352191_0003 running in uber mode : false
  14/07/14 23:55:57 INFO mapreduce.Job:  map 0% reduce 0%
  </code></pre>
  
  <blockquote>
    <p>I am using hadoop version- Hadoop 2.3.0-cdh5.0.2</p>
  </blockquote>
  
  <p>I did quick research on Google and found to increase</p>
  
  <pre><code>yarn.scheduler.minimum-allocation-mb
  yarn.nodemanager.resource.memory-mb
  </code></pre>
  
  <p>I am having single node cluster, running in my Macbook with dual core and 8 GB Ram.</p>
  
  <p>my yarn-site.xml file -</p>
  
  <pre><code>&lt;configuration&gt;
  
  &lt;!-- Site specific YARN configuration properties --&gt;
    &lt;property&gt;
    &lt;property&gt;
      &lt;name&gt;yarn.resourcemanager.hostname&lt;/name&gt;
      &lt;value&gt;resourcemanager.company.com&lt;/value&gt;
    &lt;/property&gt;   
    &lt;property&gt;
      &lt;description&gt;Classpath for typical applications.&lt;/description&gt;
      &lt;name&gt;yarn.application.classpath&lt;/name&gt;
      &lt;value&gt;
          $HADOOP_CONF_DIR,
          $HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,
          $HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,
          $HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,
          $HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*
      &lt;/value&gt;
    &lt;/property&gt;
  
    &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.local-dirs&lt;/name&gt;
      &lt;value&gt;file:///data/1/yarn/local,file:///data/2/yarn/local,file:///data/3/yarn/local&lt;/value&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.log-dirs&lt;/name&gt;
      &lt;value&gt;file:///data/1/yarn/logs,file:///data/2/yarn/logs,file:///data/3/yarn/logs&lt;/value&gt;
    &lt;/property&gt;
    &lt;property&gt;
    &lt;/property&gt;
      &lt;name&gt;yarn.log.aggregation.enable&lt;/name&gt;
      &lt;value&gt;true&lt;/value&gt; 
    &lt;property&gt;
      &lt;description&gt;Where to aggregate logs&lt;/description&gt;
      &lt;name&gt;yarn.nodemanager.remote-app-log-dir&lt;/name&gt;
      &lt;value&gt;hdfs://var/log/hadoop-yarn/apps&lt;/value&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.aux-services&lt;/name&gt;
      &lt;value&gt;mapreduce_shuffle&lt;/value&gt;
      &lt;description&gt;shuffle service that needs to be set for Map Reduce to run &lt;/description&gt;
    &lt;/property&gt;
     &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.aux-services.mapreduce.shuffle.class&lt;/name&gt;
      &lt;value&gt;org.apache.hadoop.mapred.ShuffleHandler&lt;/value&gt;
    &lt;/property&gt;
    &lt;/property&gt;
  
    &lt;property&gt;
          &lt;name&gt;yarn.app.mapreduce.am.resource.mb&lt;/name&gt;
          &lt;value&gt;8092&lt;/value&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;yarn.app.mapreduce.am.command-opts&lt;/name&gt;
          &lt;value&gt;-Xmx768m&lt;/value&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;mapreduce.framework.name&lt;/name&gt;
          &lt;value&gt;yarn&lt;/value&gt;
          &lt;description&gt;Execution framework.&lt;/description&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;mapreduce.map.cpu.vcores&lt;/name&gt;
          &lt;value&gt;4&lt;/value&gt;
          &lt;description&gt;The number of virtual cores required for each map task.&lt;/description&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;mapreduce.map.memory.mb&lt;/name&gt;
          &lt;value&gt;8092&lt;/value&gt;
          &lt;description&gt;Larger resource limit for maps.&lt;/description&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;mapreduce.map.java.opts&lt;/name&gt;
          &lt;value&gt;-Xmx768m&lt;/value&gt;
          &lt;description&gt;Heap-size for child jvms of maps.&lt;/description&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;mapreduce.jobtracker.address&lt;/name&gt;
          &lt;value&gt;jobtracker.alexjf.net:8021&lt;/value&gt;
      &lt;/property&gt;
  
   &lt;property&gt;
      &lt;name&gt;yarn.scheduler.minimum-allocation-mb&lt;/name&gt;
      &lt;value&gt;2048&lt;/value&gt;
      &lt;description&gt;Minimum limit of memory to allocate to each container request at the Resource Manager.&lt;/description&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;yarn.scheduler.maximum-allocation-mb&lt;/name&gt;
      &lt;value&gt;8092&lt;/value&gt;
      &lt;description&gt;Maximum limit of memory to allocate to each container request at the Resource Manager.&lt;/description&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;yarn.scheduler.minimum-allocation-vcores&lt;/name&gt;
      &lt;value&gt;2&lt;/value&gt;
      &lt;description&gt;The minimum allocation for every container request at the RM, in terms of virtual CPU cores. Requests lower than this won't take effect, and the specified value will get allocated the minimum.&lt;/description&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;yarn.scheduler.maximum-allocation-vcores&lt;/name&gt;
      &lt;value&gt;10&lt;/value&gt;
      &lt;description&gt;The maximum allocation for every container request at the RM, in terms of virtual CPU cores. Requests higher than this won't take effect, and will get capped to this value.&lt;/description&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.resource.memory-mb&lt;/name&gt;
      &lt;value&gt;2048&lt;/value&gt;
      &lt;description&gt;Physical memory, in MB, to be made available to running containers&lt;/description&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.resource.cpu-vcores&lt;/name&gt;
      &lt;value&gt;4&lt;/value&gt;
      &lt;description&gt;Number of CPU cores that can be allocated for containers.&lt;/description&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.aux-services&lt;/name&gt;
      &lt;value&gt;mapreduce_shuffle&lt;/value&gt;
      &lt;description&gt;shuffle service that needs to be set for Map Reduce to run &lt;/description&gt;
    &lt;/property&gt;
     &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.aux-services.mapreduce.shuffle.class&lt;/name&gt;
      &lt;value&gt;org.apache.hadoop.mapred.ShuffleHandler&lt;/value&gt;
    &lt;/property&gt;
  
  &lt;/configuration&gt;
  </code></pre>
  
  <p>my mapred-site.xml</p>
  
  <pre><code>  &lt;property&gt;    
      &lt;name&gt;mapreduce.framework.name&lt;/name&gt;    
      &lt;value&gt;yarn&lt;/value&gt;  
    &lt;/property&gt;
  </code></pre>
  
  <p>has only 1 property.
  tried several permutation and combinations but couldn't get rid of the error.</p>
  
  <p>Log of the job</p>
  
  <pre><code> 23:55:55,694 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
  2014-07-14 23:55:55,697 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
  2014-07-14 23:55:55,699 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8030
  2014-07-14 23:55:55,769 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: maxContainerCapability: 8092
  2014-07-14 23:55:55,769 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: queue: root.abhishekchoudhary
  2014-07-14 23:55:55,775 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
  2014-07-14 23:55:55,777 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-nodemanagers-proxies : 500
  2014-07-14 23:55:55,787 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1405376352191_0003Job Transitioned from INITED to SETUP
  2014-07-14 23:55:55,789 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
  2014-07-14 23:55:55,800 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1405376352191_0003Job Transitioned from SETUP to RUNNING
  2014-07-14 23:55:55,823 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1405376352191_0003_m_000000 Task Transitioned from NEW to SCHEDULED
  2014-07-14 23:55:55,824 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1405376352191_0003_m_000001 Task Transitioned from NEW to SCHEDULED
  2014-07-14 23:55:55,824 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1405376352191_0003_m_000002 Task Transitioned from NEW to SCHEDULED
  2014-07-14 23:55:55,825 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1405376352191_0003_m_000003 Task Transitioned from NEW to SCHEDULED
  2014-07-14 23:55:55,826 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1405376352191_0003_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
  2014-07-14 23:55:55,827 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1405376352191_0003_m_000001_0 TaskAttempt Transitioned from NEW to UNASSIGNED
  2014-07-14 23:55:55,827 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1405376352191_0003_m_000002_0 TaskAttempt Transitioned from NEW to UNASSIGNED
  2014-07-14 23:55:55,827 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1405376352191_0003_m_000003_0 TaskAttempt Transitioned from NEW to UNASSIGNED
  2014-07-14 23:55:55,828 INFO [Thread-49] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceReqt:8092
  2014-07-14 23:55:55,858 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1405376352191_0003, File: hdfs://localhost/tmp/hadoop-yarn/staging/abhishekchoudhary/.staging/job_1405376352191_0003/job_1405376352191_0003_1.jhist
  2014-07-14 23:55:56,773 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:4 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
  2014-07-14 23:55:56,799 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1405376352191_0003: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=&lt;memory:0, vCores:0&gt; knownNMs=1
  </code></pre>