title: Pyspark import .py file not working
body: <p>My goal is to import a custom .py file into my spark application and call some of the functions included inside that file</p>
  
  <p>Here is what I tried:</p>
  
  <p>I have a test file called <strong><em>Test.py</em></strong> which looks as follows:</p>
  
  <pre><code>def func():
      print "Import is working"
  </code></pre>
  
  <p>Inside my Spark application I do the following (as described in the docs):</p>
  
  <pre><code>sc = SparkContext(conf=conf, pyFiles=['/[AbsolutePathTo]/Test.py'])
  </code></pre>
  
  <p>I also tried this instead (after the Spark context is created):</p>
  
  <pre><code>sc.addFile("/[AbsolutePathTo]/Test.py")
  </code></pre>
  
  <p>I even tried the following when submitting my spark application:</p>
  
  <pre><code>./bin/spark-submit --packages com.datastax.spark:spark-cassandra-connector_2.10:1.5.0-M2 --py-files /[AbsolutePath]/Test.py ../Main/Code/app.py
  </code></pre>
  
  <p>However, I always get a name error:</p>
  
  <pre><code>NameError: name 'func' is not defined
  </code></pre>
  
  <p>when I am calling <strong><em>func()</em></strong> inside my <strong><em>app.py</em></strong>. (same error with 'Test' if I try to call <strong><em>Test.func()</em></strong>)</p>
  
  <p>Finally, al also tried importing the file inside the pyspark shell with the same command as above:</p>
  
  <pre><code>sc.addFile("/[AbsolutePathTo]/Test.py")
  </code></pre>
  
  <p>Strangely, I do not get an error on the import, but still, I cannot call func() without getting the error. Also, not sure if it matters, but I'm using spark locally on one machine.</p>
  
  <p>I really tried everything I could think of, but still cannot get it to work. Probably I am missing something very simple. Any help would be appreciated.</p>