title: Submit & Kill Spark Application program programmatically from another application
body: <p>I am wondering if it is possible to <strong>submit</strong>, <strong>monitor</strong> &amp; <strong>kill</strong> spark applications from another service.</p>
  
  <p>My requirements are as follows:</p>
  
  <p>I wrote a service that </p>
  
  <ol>
  <li><strong>parse</strong> user commands</li>
  <li><strong>translate</strong> them into understandable <strong>arguments</strong> to an <strong>already prepared Spark-SQL application</strong></li>
  <li>submit the application along with arguments to Spark Cluster using <strong><code>spark-submit</code></strong> from <strong><code>ProcessBuilder</code></strong></li>
  <li>And plans to run generated applications' driver in <strong>cluster mode</strong>.</li>
  </ol>
  
  <p>Other requirements needs:</p>
  
  <ul>
  <li><strong>Query</strong> about the applications <strong>status</strong>, for example, the percentage remains</li>
  <li><strong>Kill</strong> queries accrodingly</li>
  </ul>
  
  <p>What I find in <a href="http://spark.apache.org/docs/latest/spark-standalone.html" rel="nofollow noreferrer">spark standalone documentation</a> suggest kill application using:</p>
  
  <pre><code>./bin/spark-class org.apache.spark.deploy.Client kill &lt;master url&gt; &lt;driver ID&gt;
  </code></pre>
  
  <p>And should <code>find the driver ID through the standalone Master web UI at http://&lt;master url&gt;:8080.</code></p>
  
  <p>So, what am I supposed to do?</p>
  
  <p>Related SO questions:<br/>
  <a href="https://stackoverflow.com/questions/29282408/deploy-apache-spark-application-from-another-application-in-java-best-practice">Spark application finished callback</a> <br/>
  <a href="https://stackoverflow.com/questions/29282408/deploy-apache-spark-application-from-another-application-in-java-best-practice">Deploy Apache Spark application from another application in Java, best practice</a></p>