title: Passing configuration to Spark Job
body: <p>I'd like to have an external config file that I'd pass to SparkJobs. Assuming I'm running my job from an assembly and config file in on my local filesystem:</p>
  
  <pre><code>spark-class my.assembly.jar my_application.conf
  </code></pre>
  
  <p>It would be great if I could access config file in spark job, but its not possible, its main method is executed on another node.</p>
  
  <p>I've been trying to use --files argument for spark-class, but this does not seem to work. </p>
  
  <p>Similar behavior (to --files) tried in spark repl ends with error:</p>
  
  <pre><code>val inFile = sc.textFile(SparkFiles.get("conf.a"))
  inFile.first()
  </code></pre>
  
  <p>The above assumes file <em>conf.a</em> has been passed to spark-class with --files option.</p>
  
  <p>Any thoughts on this problem? How to fix the issue? I'd really like to use external file as configuration source.</p>
  
  <p>I'm using apache-spark-0.9.0</p>