title: How to get execution DAG from spark web UI after job has finished running, when I am running spark on YARN?
body: <p>I frequently do analysis of the DAG of my spark job while it is running.  But, it is annoying to have to sit and watch the application while it is running in order to see the DAG.</p>
  
  <p>So, I tried to view the DAg using this thing called the <code>spark history-server</code>, which I know should help me see past jobs.  I'm easily able to access port <code>18080</code>, and I can see the history server UI.</p>
  
  <p>But, it doesn't show me any information related to the spark program's execution.  I know I have the history server running, because when I do <code>sudo service --status-all</code> I see</p>
  
  <p><code>spark history-server is running            [ OK ]</code></p>
  
  <p>So I already tried what this question suggested: <a href="https://stackoverflow.com/questions/39936593/cannot-see-finished-job-in-spark-web-ui">here</a>.</p>
  
  <p>I think this is because I'm running spark on YARN, and it can only use one resource manager at a time? maybe?  </p>
  
  <p>So, how do I see the spark execution DAG,  <em>*after*</em> a job has finished?  and more specifically, when running YARN as my resource manager?</p>