title: Provide Maven dependencies while launcing Spark application programatically
body: <p>I want to launch multiple Spark streams programmatically those depend on external jar files.</p>
  
  <p>First approach I tried was to execute "SparkSubmit.main(argument)" statement from within my Java program where "argument" is a String array consisting of parameters such as --master, --class. This worked fine, but it was able to launch only a single Stream.</p>
  
  <p>1st approach code snippet:</p>
  
  <p>String command = buildCommand();</p>
  
  <p>SparkSubmit.main(command);</p>
  
  <p>Value of "command" would be something similar to 
  --master [master] --class [StreamingClass] --repositories [Maven Repository] --packages [lot of Maven coordinates] [Jar File Name] [Required Command Line Parameters]</p>
  
  <p>Next approach was to try ProcessBuilder to spawn multiple threads; with each thread launching a separate "SparkSubmit.main(command)" . This did not work. I could see process ids on Unix, but for some reason the data received on Kafka was not processed.</p>
  
  <p>Third approach was to try SparkLauncher. However, I am a at a loss how to specify all these external jar files in form of Maven coordinates and repository using SparkLauncher. SparkLauncher.EXECUTER_EXTRA_CLASSPATH cannot be used as I don't want to specify classpath. SparkLauncher.EXECUTOR_EXTRA_JAVA_OPTIONS may be used, but not sure what keys should be used to specify repository and packages. (--repositories and --packages or something else).</p>
  
  <p>Don't want to create a uber jar as its size is over 50M.</p>
  
  <p>Your help is really appreciated. </p>
  
  <p>Thanks a lot in advance.</p>
  
  <p>--Shri</p>