title: How do I install numpy and pandas for Python 3.5 in Spark?
body: <p>I am trying to run a linear regression in Spark using Python 3.5 instead of Python 2.7.  So first I exported PYSPARK_PHTHON=python3.  I received an error "No module named numpy".  I tried to "pip install numpy" but pip doesn't recognize the setting PYSPARK_PYTHON.  How to I ask pip to install numpy for 3.5?  Thank you ...</p>
  
  <pre><code>$ export PYSPARK_PYTHON=python3
  
  $ spark-submit linreg.py
  ....
  Traceback (most recent call last):
    File "/home/yoda/Code/idenlink-examples/test22-spark-linreg/linreg.py", line 115, in &lt;module&gt;
  from pyspark.ml.linalg import Vectors
    File "/home/yoda/install/spark/python/lib/pyspark.zip/pyspark/ml/__init__.py", line 22, in &lt;module&gt;
    File "/home/yoda/install/spark/python/lib/pyspark.zip/pyspark/ml/base.py", line 21, in &lt;module&gt;
    File "/home/yoda/install/spark/python/lib/pyspark.zip/pyspark/ml/param/__init__.py", line 26, in &lt;module&gt;
    ImportError: No module named 'numpy'
  
  $ pip install numpy
  Requirement already satisfied: numpy in /home/yoda/.local/lib/python2.7/site-packages
  
  $ pyspark
  Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
  [GCC 5.4.0 20160609] on linux
  Type "help", "copyright", "credits" or "license" for more information.
  17/02/09 20:29:20 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  17/02/09 20:29:20 WARN Utils: Your hostname, yoda-VirtualBox resolves to a loopback address: 127.0.1.1; using 10.0.2.15 instead (on interface enp0s3)
  17/02/09 20:29:20 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
  17/02/09 20:29:31 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
  Welcome to
        ____              __
       / __/__  ___ _____/ /__
      _\ \/ _ \/ _ `/ __/  '_/
     /__ / .__/\_,_/_/ /_/\_\   version 2.1.0
        /_/
  
  Using Python version 3.5.2 (default, Nov 17 2016 17:05:23)
  SparkSession available as 'spark'.
  &gt;&gt;&gt; import site; site.getsitepackages()
  ['/usr/local/lib/python3.5/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.5/dist-packages']
  &gt;&gt;&gt; 
  </code></pre>