Exception: Python in worker has different version 2.7 than that in driver 2.6, PySpark cannot run with different minor versions
Install at least Python 2.7 on each node and configure SPARK_PYTHON environment variable to point to the required installation. Spark doesn't support mixed environments and doesn't support Python 2.6 anymore.
We have a hadoop cluster of 625 nodes. But some of them are centos 6 (python 2.6) and some are centos 7 (python 2.7). So how can I resolve this as I am getting this error constantly.