title: Submit Spark Application on Kubernetes in Cluster mode : Configured service account doesn't have access
body: <p>I try to submit a Spark application to a Kubernetes cluster (Minikube).
  When running my spark submit in client mode, everything goes well. 3 executors are created in 3 pods, and the code is executed. Here is my submit command : </p>
  
  <pre><code>[MY_PATH]/bin/spark-submit \
     --master k8s://https://[API_SERVER_IP]:8443 \
     --deploy-mode client \
     --name [Name] \
     --class [MyClass] \
     --conf spark.kubernetes.container.image=spark:2.4.0 \
     --conf spark.executor.instances=3 \
     [PATH/TO/MY/JAR].jar
  </code></pre>
  
  <p>Now, I adapted it to run in cluster mode : </p>
  
  <pre><code>[MY_PATH]/bin/spark-submit \
     --master k8s://https://[API_SERVER_IP]:8443 \
     --deploy-mode cluster \
     --name [Name] \
     --class [MyClass] \
     --conf spark.kubernetes.container.image=spark:2.4.0 \
     --conf spark.executor.instances=3 \
     local://[PATH/TO/MY/JAR].jar
  </code></pre>
  
  <p>This time, a driver pod is created as well as a driver service, and then the driver pod fail. On the Kubernetes I can see the following error : </p>
  
  <pre><code>MountVolume.SetUp failed for volume "spark-conf-volume" : configmap "sparkpi-1555314081444-driver-conf-map" not found
  </code></pre>
  
  <p>And in the pod logs I have the error : </p>
  
  <pre><code>Forbidden!Configured service account doesn't have access. 
  Service account may have been revoked. 
  pods "sparkpi-1555314081444-driver" is forbidden: User "system:serviceaccount:default:default" cannot get resource "pods" in API group "" in the namespace "default".
  </code></pre>
  
  <p>Here is the full stacktrace :</p>
  
  <pre><code>org.apache.spark.SparkException: External scheduler cannot be instantiated
      at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2794)
      at org.apache.spark.SparkContext.&lt;init&gt;(SparkContext.scala:493)
      at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
      at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:935)
      at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:926)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:926)
      at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:31)
      at org.apache.spark.examples.SparkPi.main(SparkPi.scala)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
      at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
      at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
      at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
      at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) 
      at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
      at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
      at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://kubernetes.default.svc/api/v1/namespaces/default/pods/sparkpi-1555314081444-driver. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods "sparkpi-1555314081444-driver" is forbidden: User "system:serviceaccount:default:default" cannot get resource "pods" in API group "" in the namespace "default".
      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:470)
      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:407)
      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:379)
      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:343)
      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:312)
      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:295)
      at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:783)
      at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:217)
      at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:184)
      at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator$$anonfun$1.apply(ExecutorPodsAllocator.scala:57)
      at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator$$anonfun$1.apply(ExecutorPodsAllocator.scala:55)
      at scala.Option.map(Option.scala:146)
      at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.&lt;init&gt;(ExecutorPodsAllocator.scala:55)
      at org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:89)
      at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2788)
      ... 20 more 
  </code></pre>
  
  <p>What should I do to make it work ?</p>