title: Redirecting to log server for container when view logs of a completed spark jobs run on yarn
body: <p>I'm running spark on yarn.</p>
  
  <p>My spark versoin is 2.1.1, and hadoop version is apache hadoop 2.7.3.</p>
  
  <p>when a spark job running on yarn in cluster mode, I can view the Executor's log via the stdout/stderr links like </p>
  
  <pre><code>http://hadoop-slave1:8042/node/containerlogs/container_1500432603585_0148_01_000001/hadoop/stderr?start=-4096
  </code></pre>
  
  <p>but when the job completed, view the Executor's log via the stdout/stderr links will get an error page like </p>
  
  <blockquote>
    <p>Redirecting to log server for container_1500432603585_0148_01_000001</p>
    
    <p>java.lang.Exception: Unknown container. Container either has not
    started  or has already completed or doesn't belong to this node at
    all.</p>
  </blockquote>
  
  <p>And then it will auto redirect to </p>
  
  <pre><code>http://hadoop-slave1:8042/node/hadoop-master:19888/jobhistory/logs/hadoop-slave1:36207/container_1500432603585_0148_01_000001/container_1500432603585_0148_01_000001/hadoop
  </code></pre>
  
  <p>and get other error page like</p>
  
  <pre><code>Sorry, got error 404  
  Please consult RFC 2616 for meanings of the error code.  
  
  Error Details
  
  org.apache.hadoop.yarn.webapp.WebAppException: /hadoop-master:19888/jobhistory/logs/hadoop-slave1:50284/container_1500432603585_0145_01_000002/container_1500432603585_0145_01_000002/oryx: controller for hadoop-master:19888 not found
          at org.apache.hadoop.yarn.webapp.Router.resolveDefault(Router.java:232)
          at org.apache.hadoop.yarn.webapp.Router.resolve(Router.java:140)
          at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:134)
          at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
          at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
  </code></pre>
  
  <p>Actually i can visit the Executor's log using this url when the
   spark job completed:</p>
  
  <pre><code>http://hadoop-master:19888/jobhistory/logs/hadoop-slave1:36207/container_1500432603585_0148_01_000001/container_1500432603585_0148_01_000001/hadoop
  </code></pre>
  
  <p>it's a little different from the previous url, it remove the head "hadoop-slave1:8042/node/".</p>
  
  <p>Does anyone knows another better method to view the spark logs when the spark job completed ?</p>
  
  <p>I have configed the yarn-site.xml </p>
  
  <pre><code>  &lt;property&gt;
        &lt;name&gt;yarn.resourcemanager.hostname&lt;/name&gt;
        &lt;value&gt;hadoop-master&lt;/value&gt;
        &lt;description&gt;The hostname of the RM.&lt;/description&gt;
    &lt;/property&gt;  
  
    &lt;property&gt;
        &lt;name&gt;yarn.log-aggregation-enable&lt;/name&gt;
        &lt;value&gt;true&lt;/value&gt;
    &lt;/property&gt;
  
    &lt;property&gt;
        &lt;name&gt;yarn.log.server.url&lt;/name&gt;
        &lt;value&gt;${yarn.resourcemanager.hostname}:19888/jobhistory/logs&lt;/value&gt;
    &lt;/property&gt;
  </code></pre>
  
  <p>and mapred-site.xml</p>
  
  <pre><code>&lt;property&gt;
      &lt;name&gt;mapreduce.jobhistory.address&lt;/name&gt;
      &lt;value&gt;${yarn.resourcemanager.hostname}:10020&lt;/value&gt;
    &lt;/property&gt;
  
    &lt;property&gt;
      &lt;name&gt;mapreduce.jobhistory.admin.address &lt;/name&gt;
      &lt;value&gt;${yarn.resourcemanager.hostname}:10033&lt;/value&gt;
    &lt;/property&gt;
  
    &lt;property&gt;
      &lt;name&gt;mapreduce.jobhistory.webapp.address&lt;/name&gt;
      &lt;value&gt;${yarn.resourcemanager.hostname}:19888&lt;/value&gt;
    &lt;/property&gt;
  </code></pre>