title: Submitting spark app as a yarn job from Eclipse and Spark Context
body: <p>I can already submit <code>local</code> spark jobs (written in Scala) from my Eclipse IDE. However, I would like to modify my Spark context (inside my application) so that when I 'Run' the app (inside Eclipse),  the job will be sent to my remote cluster using Yarn as a resource manager. </p>
  
  <p>Using <code>spark-submit</code>, I can successfully submit the job to the cluster as:
  <code>spark-submit --class &lt;main class&gt; --master yarn-cluster &lt;jar&gt;</code></p>
  
  <p>I want to achieve the same result inside the IDE. 
  My sbt config (app root directory) looks like: 
  <code>
  libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.1" 
  libraryDependencies += "org.apache.spark" %% "spark-yarn" % "1.5.1" % "provided"
  </code>
  Inside my app: 
  <code>
  val conf = new SparkConf().setAppName("xxx").setMaster("yarn-cluster")
  </code>
  However, I am getting the following errors:</p>
  
  <p><code>
   Detected yarn-cluster mode, but isn't running on a cluster. Deployment to YARN is not supported directly by SparkContext. Please use spark-submit.
  </code></p>