title: How to get return code from spark-submit?
body: <p>I am trying to execute a spark job using an ssh connection on a remote location.  </p>
  
  <p>There have been instances where the job failed but the scheduler marked it as "success" so i want to check the return code of spark-submit so i could forcefully fail it.</p>
  
  <p>Below is the code I'm using</p>
  
  <pre class="lang-py prettyprint-override"><code>def execute_XXXX():
      f = open('linux.pem','r')
      s = f.read()
      keyfile = StringIO.StringIO(s)
      mykey = paramiko.RSAKey.from_private_key(keyfile)
      sshcon   = paramiko.SSHClient()
      sshcon.set_missing_host_key_policy(paramiko.AutoAddPolicy())
      sshcon.connect('XXX.XXX.XXX.XXX', username='XXX', pkey=mykey)
      stdin, stderr, stdout= sshcon.exec_command('spark-submit XXX.py')
  
      logger.info("XXX ------&gt;"+str(stdout.readlines()))
      logger.info("Error---------&gt;"+str(stderr.readlines()))
  </code></pre>
  
  <p>How do I get the return code for the spark-submit job so I can forcefully fail the task.
  Or could you suggest an alternate solution.</p>
  
  <p>Thanks, Chetan</p>