title: Submitting to a Spark Master inside a kubernetes cluster
body: <p>I have a Kubernetes cluster composed of only one VM (minikube cluster).</p>
  
  <p>On this cluster, I have a Spark Master and two Workers running. I have set up the Ingress addon in the following way (My spark components use the default ports) : </p>
  
  <pre><code>apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    name: minikube-ingress
    annotations:
  spec:
    rules:
    - host: spark-kubernetes
      http:
        paths:
        - path: /web-ui
          backend:
            serviceName: spark-master
            servicePort: 8080
        - path: /
          backend:
            serviceName: spark-master
            servicePort: 7077
  </code></pre>
  
  <p>And I added my k8s IP in my <code>/etc/hosts</code></p>
  
  <pre><code>[MINIKUBE_IP] spark-kubernetes
  </code></pre>
  
  <p>I am able to connect to the Master webui through <code>http://spark-kubernetes/web-ui</code> : 
  <a href="https://i.stack.imgur.com/rz8eb.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/rz8eb.png" alt="enter image description here"></a></p>
  
  <p>I now want to submit a JAR stored on my local machine (the spark-examples for example). I expected this command to work : </p>
  
  <pre><code>./bin/spark-submit \
      --master spark://spark-kubernetes \
      --deploy-mode cluster \
      --class org.apache.spark.examples.SparkPi \
       ./examples/jars/spark-examples_2.11-2.4.0.jar
  </code></pre>
  
  <p>But I get the following error : </p>
  
  <pre><code>2019-04-04 08:52:36 WARN  SparkSubmit$$anon$2:87 - Failed to load .
  java.lang.ClassNotFoundException: 
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:348)
      at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
      at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:810)
      at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
      at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
      at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
      at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
      at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
      at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  </code></pre>
  
  <p>What have I done wrong?</p>
  
  <p>Note : </p>
  
  <ul>
  <li>I know that with Spark 2.4 I can have a cluster without a Master and submit directly to the k8s, but I want to do it with a master for now</li>
  <li>I use Spark 2.4</li>
  <li>I use Kubernetes 1.14</li>
  </ul>