title: Hadoop Yarn container logs missing
body: <p>We usually will be able to see yarn container logs in "/var/log/hadoop-yarn/containers" path. Though I am able to see logs for successful jobs, I am not able to see the logs for failed jobs. The node manager logs shows the logs getting deleted. </p>
  
  <p>Log:</p>
  
  <pre><code>2017-07-13 14:16:04,170 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor (DeletionService #1): Deleting path : /var/log/hadoop-yarn/containers/application_1234567890_12345/container_11234567890_12345_11_0000
  01/stdout
  2017-07-13 14:16:04,180 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl (LogAggregationService #6093): renaming /var/log/hadoop-yarn/apps/hadoop/logs/application_1234567890_12345/xx.xx.xx.xx_8041.tmp to /var/log/hadoop-yarn/apps/hadoop/logs/application_1234567890_12345/xx.xx.xx.xx_8041
  2017-07-13 14:16:04,181 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor (DeletionService #3): Deleting path : /var/log/hadoop-yarn/containers/application_1234567890_12345
  2017-07-13 14:16:06,048 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl (Container Monitor): Stopping resource-monitoring for container_11234567890_12345_11_0000
  </code></pre>
  
  <p>Here's a snippet of my yarn-site.xml.</p>
  
  <p>Can some one please advise on what config needs to be modified to retain logs for failed jobs?</p>
  
  <pre><code>&lt;property&gt;
      &lt;name&gt;yarn.log-aggregation-enable&lt;/name&gt;
      &lt;value&gt;true&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;name&gt;yarn.log.server.url&lt;/name&gt;
      &lt;value&gt;http://ip-XX.XX.XX.XX:19888/jobhistory/logs&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.local-dirs&lt;/name&gt;
      &lt;value&gt;/mnt/yarn&lt;/value&gt;
      &lt;final&gt;true&lt;/final&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;description&gt;Where to store container logs.&lt;/description&gt;
      &lt;name&gt;yarn.nodemanager.log-dirs&lt;/name&gt;
      &lt;value&gt;/var/log/hadoop-yarn/containers&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;description&gt;Where to aggregate logs to.&lt;/description&gt;
      &lt;name&gt;yarn.nodemanager.remote-app-log-dir&lt;/name&gt;
      &lt;value&gt;/var/log/hadoop-yarn/apps&lt;/value&gt;
  &lt;/property&gt;
  
  
  &lt;property&gt;
      &lt;name&gt;yarn.log-aggregation.enable-local-cleanup&lt;/name&gt;
      &lt;value&gt;true&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;name&gt;yarn.scheduler.increment-allocation-mb&lt;/name&gt;
      &lt;value&gt;32&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;name&gt;yarn.log-aggregation.retain-seconds&lt;/name&gt;
      &lt;value&gt;604800&lt;/value&gt;
  &lt;/property&gt;
  </code></pre>