title: Spark on kubernetes with zeppelin
body: <p>I am following this guide to run up a zeppelin container in a local kubernetes cluster set up using minikube.</p>
  
  <p><a href="https://zeppelin.apache.org/docs/0.9.0-SNAPSHOT/quickstart/kubernetes.html" rel="nofollow noreferrer">https://zeppelin.apache.org/docs/0.9.0-SNAPSHOT/quickstart/kubernetes.html</a></p>
  
  <p>I am able to set up zeppelin and run some sample code there. I have downloaded spark 2.4.5 &amp; 2.4.0 source code and built it for kubernetes support with the following command:</p>
  
  <pre><code>./build/mvn -Pkubernetes -DskipTests clean package
  </code></pre>
  
  <p>Once spark is built I created a docker container as explained in the article:</p>
  
  <pre><code>bin/docker-image-tool.sh -m -t 2.4.X build
  </code></pre>
  
  <p>I configured zeppelin to use the spark image which was built with kubernetes support. The article above explains that the spark interpreter will auto configure spark on kubernetes to run in client mode and run the job.</p>
  
  <p>But whenever I try to run any parahgraph with spark I receive the following error</p>
  
  <pre><code>Exception in thread "main" java.lang.IllegalArgumentException: basedir must be absolute: ?/.ivy2/local
  </code></pre>
  
  <p>I tried setting the spark configuration <code>spark.jars.ivy</code> in zeppelin to point to a temp directory but that does not work either. </p>
  
  <p>I found a similar issue here:
  <a href="https://stackoverflow.com/questions/50861477/basedir-must-be-absolute-ivy2-local">basedir must be absolute: ?/.ivy2/local</a></p>
  
  <p>But I can't seem to configure spark to run with the <code>spark.jars.ivy /tmp/.ivy</code> config. I tried building spark with the <strong>spark-defaults.conf</strong> when building spark but that does not seems to be working either.</p>
  
  <p>Quite stumped at this problem and how to solve it any guidance would be appreciated.</p>
  
  <p>Thanks!</p>