title: Spark on K8's Issues loading jar
body: <p>I am trying to run a sample spark application(provided in the spark examples jar) on kubernetes and trying to understand the behavior. In this process, I did the following,</p>
  
  <ol>
  <li>Built a running kubernetes cluster with 3 nodes (1 master and 2 child) with adequate resources(10 cores, 64Gigs mem, 500GB disk). Note that I don't have internet access on my nodes.</li>
  <li>Installed Spark distribution - spark-2.3.3-bin-hadoop2.7</li>
  <li>As there is no internet access on the node, I preloaded a spark image( from gcr.io/cloud-solutions-images/spark:v2.3.0-gcs) into the docker on the node running kubernetes master</li>
  <li>Running spark-submit to k8's as follows,</li>
  </ol>
  
  <pre><code>./bin/spark-submit --master k8s://https://test-k8:6443 \
                     --deploy-mode cluster \ 
                     --name spark-pi \
                     --class org.apache.spark.examples.SparkPi \
                     --conf spark.executor.instances=5 \
                     --conf spark.kubernetes.container.image=gcr.io/cloud-solutions-images/spark:v2.3.0-gcs \
                     --conf spark.kubernetes.driver.pod.name=spark-pi-driver \
                     --conf spark.kubernetes.container.image.pullPolicy=IfNotPresent \
                     local:///opt/spark/examples/jars/spark-examples_2.11-2.3.3.jar
  </code></pre>
  
  <p>However, it fails with the below error,</p>
  
  <pre><code>Error: Could not find or load main class org.apache.spark.examples.SparkPi
  </code></pre>
  
  <p>In regards to the above I have below questions:</p>
  
  <ol>
  <li>Do we need to provide Kubernetes a distribution of spark? and is that what we are doing with?</li>
  </ol>
  
  <pre><code>--conf spark.kubernetes.container.image=gcr.io/cloud-solutions-images/spark:v2.3.0-gcs
  
  </code></pre>
  
  <ol start="2">
  <li>If I have my own spark example, for say processing events from Kafka. What should be my approach?</li>
  </ol>
  
  <p>Any help in debugging the above Error and answering my follow up questions is thankful.</p>