title: How to access Spark web UI in Yarn mode
body: <p>I am running a pyspark job as a AWS EMR step and the script takes well over 15 minutes to run. I have 1 master and 3 core nodes in the EMR cluster. I want to find out why and which part of my script is taking long. For that I wanted to see the Spark web UI. When I click on "Tracking URL: Applicationmaster" in Yarn UI (port 8088), my browser keeps spinning and it unable to display the Spark UI. The URL link in the browser is:</p>
  
  <pre><code>http://ip-172-31-x-x.ec2.internal:20888/proxy/application_1579701541309_1029/
  </code></pre>
  
  <p>This is obviously private DNS. How do I see the Spark UI even if this is on a temporary basis for me to troubleshoot. I can change the AWS Security group, if needed. And later, how can this be handled when I am in production environment?</p>
  
  <p>Thanks</p>