title: Spark unable to find "spark-version-info.properties" when run from ammonite script
body: <p>I have an ammonite script which creates a spark context:</p>
  
  <pre><code>#!/usr/local/bin/amm
  
  import ammonite.ops._
  
  import $ivy.`org.apache.spark:spark-core_2.11:2.0.1`
  
  import org.apache.spark.{SparkConf, SparkContext}
  
  @main
  def main(): Unit = {
    val sc =  new SparkContext(new SparkConf().setMaster("local[2]").setAppName("Demo"))
  }
  </code></pre>
  
  <p>When I run this script, it throws an error:</p>
  
  <pre><code>Exception in thread "main" java.lang.ExceptionInInitializerError
  Caused by: org.apache.spark.SparkException: Error while locating file spark-version-info.properties
  ...
  Caused by: java.lang.NullPointerException
      at java.util.Properties$LineReader.readLine(Properties.java:434)
      at java.util.Properties.load0(Properties.java:353)
  </code></pre>
  
  <p>The script isn't being run from the spark installation directory and doesn't have any knowledge of it or the resources where this version information is packaged - it only knows about the ivy dependencies. So perhaps the issue is that this resource information isn't on the classpath in the ivy dependencies. I have seen other spark "standalone scripts" so I was hoping I could do the same here.</p>
  
  <hr>
  
  <p>I poked around a bit to try and understand what was happening. I was hoping I could programmatically hack some build information into the system properties at runtime.</p>
  
  <p>The source of the exception comes from <a href="https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/package.scala" rel="nofollow">package.scala</a> in the spark library. The relevant bits of code are</p>
  
  <pre><code>val resourceStream = Thread.currentThread().getContextClassLoader.
    getResourceAsStream("spark-version-info.properties")
  
  try {
    val unknownProp = "&lt;unknown&gt;"
    val props = new Properties()
    props.load(resourceStream) &lt;--- causing a NPE?
    (
      props.getProperty("version", unknownProp),
      // Load some other properties
    )
  } catch {
    case npe: NullPointerException =&gt;
      throw new SparkException("Error while locating file spark-version-info.properties", npe)
  </code></pre>
  
  <p>It seems that the implicit assumption is that <code>props.load</code> will fail with a NPE if the version information can't be found in the resources. (That's not so clear to the reader!)</p>
  
  <p>The NPE itself looks like it's coming from this code in <code>java.util.Properties.java</code>:</p>
  
  <pre><code>class LineReader {
      public LineReader(InputStream inStream) {
          this.inStream = inStream;
          inByteBuf = new byte[8192];
      }
  
      ...
      InputStream inStream;
      Reader reader;
  
      int readLine() throws IOException {
        ...
        inLimit = (inStream==null)?reader.read(inCharBuf)
                                  :inStream.read(inByteBuf);
  </code></pre>
  
  <p>The <code>LineReader</code> is constructed with a null <code>InputStream</code> which the class internally interprets as meaning that the <code>reader</code> is non-null and should be used instead - but it's also <code>null</code>. (Is this kind of stuff really in the standard library? Seems very unsafe...)</p>
  
  <hr>
  
  <p>From looking at the <code>bin/spark-shell</code> that comes with spark, it adds <code>-Dscala.usejavacp=true</code> when it launches <code>spark-submit</code>. Is this the right direction?</p>
  
  <p>Thanks for your help!</p>