title: SparkLauncher not launching application
body: <p>I have written a small class deployed on a unix machine and I can't seem to figure out why this error this coming. I checked my <code>SPARK_HOME</code> and added all required options as shown in the class below. I have trying to write this as way to monitor spark threads running eventually. Spark-submit works perfectly so I know the setup of the enviroment is not the issue.</p>
  
  <pre><code>    package com.james.SparkLauncher2;
      import java.io.IOException;
      import java.util.HashMap;
      import java.util.Map;
  
      import org.apache.log4j.Logger;
      import org.apache.spark.launcher.SparkAppHandle;
      import org.apache.spark.launcher.SparkLauncher;
      public class SparkLauncher2 
  
      {
          static final Logger LOGGER = Logger.getLogger(SparkLauncher2.class);
          public static void main(String[] args) {
              try {
                  LOGGER.info("In main of SparkLauncher2");
                  Map &lt;String, String&gt; env= new HashMap&lt;&gt;();
                  env.put( "SPARK_HOME", "/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/spark");
                  env.put(" SPARK_LIBRARY_PATH", "/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/spark/lib");
                  System.out.println("Environments setup correctly");
  
  
                  //pass in enviroment variables  
                  SparkAppHandle sparkLauncher= new SparkLauncher(env)
                  .setAppResource("/home/james/fe.jar")
                  //This conf file works well with the spark submit so it shouldn't be source of the issue
                  .setPropertiesFile("/etc/spark/conf/spark-defaults.conf")
                  .setMainClass("com.james.SparkLauncher2.SparkLauncher2")
                  .setConf(SparkLauncher.DRIVER_MEMORY, "2g")
                  .setDeployMode("client")
                  .setVerbose(true)
                  .setConf("spark.yarn.keytab ","/home/james/my.keytab")
                  .setConf("spark.yarn.principal","somestring")
                  .setConf("spark.app.name ","SparkLauncher2") //add class name for example HbaseTest
                  .setConf("spark.jars","/home/james/flume-test.jar,/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/bin/test")
                  //call listener class to see if there is any state change
                  .startApplication(new MyListener());
  
                  sparkLauncher.stop();
  
                  //handle.stop();
              } catch (IOException e) {
                  e.printStackTrace();
              }
              //this exception is what gets thrown
              catch(Exception e){
                  LOGGER.info("General exception");
                  e.printStackTrace();
              }
          }
      }
  </code></pre>
  
  <p>I inteneded this class for mostly a check of change of state but no state change is logged</p>
  
  <pre><code>        class MyListener implements SparkAppHandle.Listener {
              @Override
              public void stateChanged(SparkAppHandle handle) {
                  System.out.println("state changed " + handle.getState());
              }
  
              @Override
              public void infoChanged(SparkAppHandle handle) {
                  System.out.println("info changed " + handle.getState());
              }
          }
  </code></pre>
  
  <p>Here is the exception I checked the directories and all of them seem to correct. I even wrote an alternate version where everything was hard coded into the setConf method. obviously no spark job is started. I don't see any jobs on the UI as well. CommandBuilder class documentation is not clear on how this exception will be thrown. For context purposes this is Java 7 and spark 1.6</p>
  
  <pre><code>    java.lang.IllegalStateException: Application is still not connected.
              at org.apache.spark.launcher.CommandBuilderUtils.checkState(CommandBuilderUtils.java:249)
              at org.apache.spark.launcher.ChildProcAppHandle.stop(ChildProcAppHandle.java:74)
              at com.james.SparkLauncher2.SparkLauncher2.main(SparkLauncher2.java:43)
  </code></pre>