title: Yarn resource manager fails the application even though Spark application executes successfully
body: <p>I am running simple hello world python script with AWS EMR + Spark + Yarn.</p>
  
  <p>Looking at the logs, even though the Spark application succeeds, overall job is marked as failed by Yarn resource manager.</p>
  
  <ul>
  <li>Logs for the spark application shows success. "Hello world" is printed in stdout also. (See <a href="https://pastebin.ca/3941319" rel="nofollow noreferrer">pastebin</a> for application logs)</li>
  <li>Logs for the node manager show no issue or error. (See <a href="https://pastebin.ca/3941320" rel="nofollow noreferrer">pastebin</a> for node manager logs)</li>
  <li>Logs for the resource manager on master host show that resource manager marks the application as FAILED even though application completion seems to be successful. There is no apparent reason in the log for the failure! (See <a href="https://pastebin.ca/3941332" rel="nofollow noreferrer">pastebin</a> for resource manager logs)</li>
  </ul>
  
  <p>I checked all logs and cannot really figure out the root cause. What could be the issue? How can I debug further?</p>