title: How to set Master address for Spark examples from command line
body: <p><strong>NOTE:</strong> <em>They author is looking for answers to set the Spark Master when running Spark examples that involves <strong>no</strong> changes to the source code, but rather only options that can be done from the command-line if at all possible.</em></p>
  
  <p>Let us consider the run() method of the BinaryClassification example:</p>
  
  <pre><code>  def run(params: Params) {
      val conf = new SparkConf().setAppName(s"BinaryClassification with $params")
      val sc = new SparkContext(conf)
  </code></pre>
  
  <p>Notice that the SparkConf did not provide any means to configure the SparkMaster.</p>
  
  <p>When running this program from Intellij with the following arguments:</p>
  
  <pre><code>--algorithm LR --regType L2 --regParam 1.0 data/mllib/sample_binary_classification_data.txt
  </code></pre>
  
  <p>the following error occurs:</p>
  
  <pre><code>Exception in thread "main" org.apache.spark.SparkException: A master URL must be set
  in your configuration
      at org.apache.spark.SparkContext.&lt;init&gt;(SparkContext.scala:166)
      at org.apache.spark.examples.mllib.BinaryClassification$.run(BinaryClassification.scala:105)
  </code></pre>
  
  <p>I have also tried adding in the Spark Master url anyways (though the code seems NOT to support it ..)</p>
  
  <pre><code>  spark://10.213.39.125:17088   --algorithm LR --regType L2 --regParam 1.0 
    data/mllib/sample_binary_classification_data.txt
  </code></pre>
  
  <p>and</p>
  
  <pre><code>--algorithm LR --regType L2 --regParam 1.0 spark://10.213.39.125:17088
  data/mllib/sample_binary_classification_data.txt
  </code></pre>
  
  <p>Both do not work with error:</p>
  
  <pre><code>Error: Unknown argument 'data/mllib/sample_binary_classification_data.txt'
  </code></pre>
  
  <p>For reference here is the options parsing - which does nothing with SparkMaster:</p>
  
  <pre><code>val parser = new OptionParser[Params]("BinaryClassification") {
    head("BinaryClassification: an example app for binary classification.")
    opt[Int]("numIterations")
      .text("number of iterations")
      .action((x, c) =&gt; c.copy(numIterations = x))
    opt[Double]("stepSize")
      .text(s"initial step size, default: ${defaultParams.stepSize}")
      .action((x, c) =&gt; c.copy(stepSize = x))
    opt[String]("algorithm")
      .text(s"algorithm (${Algorithm.values.mkString(",")}), " +
      s"default: ${defaultParams.algorithm}")
      .action((x, c) =&gt; c.copy(algorithm = Algorithm.withName(x)))
    opt[String]("regType")
      .text(s"regularization type (${RegType.values.mkString(",")}), " +
      s"default: ${defaultParams.regType}")
      .action((x, c) =&gt; c.copy(regType = RegType.withName(x)))
    opt[Double]("regParam")
      .text(s"regularization parameter, default: ${defaultParams.regParam}")
    arg[String]("&lt;input&gt;")
      .required()
      .text("input paths to labeled examples in LIBSVM format")
      .action((x, c) =&gt; c.copy(input = x))
  </code></pre>
  
  <p>So .. yes .. I could go ahead and modify the source code.  But I suspect instead I am missing an <em>available</em> tuning knob to make this work that does not involve modifying the source code.</p>