title: cannot find python3.7 when runing spark using spark-submit?
body: <p>There is a problem that has bothered me for a few days. It occurs when I am running Spark application using spark-submit, it shows like this below:</p>
  
  <pre><code>09:14:36 Exception in thread "main" java.io.IOException: Cannot run program "python3.7": error=2, No such file or directory
  09:14:36    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
  09:14:36    at org.apache.spark.deploy.PythonRunner$.main(PythonRunner.scala:100)
  09:14:36    at org.apache.spark.deploy.PythonRunner.main(PythonRunner.scala)
  09:14:36    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  09:14:36    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  09:14:36    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  09:14:36    at java.lang.reflect.Method.invoke(Method.java:498)
  09:14:36    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
  09:14:36    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
  09:14:36    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
  09:14:36    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
  09:14:36    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
  09:14:36    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  09:14:36 Caused by: java.io.IOException: error=2, No such file or directory
  09:14:36    at java.lang.UNIXProcess.forkAndExec(Native Method)
  09:14:36    at java.lang.UNIXProcess.&lt;init&gt;(UNIXProcess.java:247)
  09:14:36    at java.lang.ProcessImpl.start(ProcessImpl.java:134)
  09:14:36    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
  09:14:36    ... 12 more
  </code></pre>
  
  <p>I have configured all the environment Variables both in <code>~/.bashrc</code> and <code>/etc/profile</code>. And I am absolutely sure that all the environment variables are correct. And I can run <code>python3.7</code> or PySpark in my Ubuntu.</p>
  
  <pre><code>export JAVA_HOME=/usr/lib/jvm/java-8-oracle
  export PATH=${PATH}:${JAVA_HOME}
  export PATH=${PATH}:/mwdata/python3.7/bin
  export SPARK_HOME=/srv/spark
  export PATH=${PATH}:${SPARK_HOME}/bin
  export PYSPARK_PYTHON=python3.7
  export PYSPARK_DRIVER_PYTHON=python3.7
  </code></pre>
  
  <p>So why doesn't it find <code>python3.7</code>?</p>