title: Unable to import pyspark in dataproc cluster on GCP
body: <p>I just setup a cluster on Google Cloud Platform to run some pyspark jobs.
  Initially I used ipython.sh (from the github repository) as initialization script for the cluster. This allowed the cluster to startup nicely, however when trying to import pyspark in an Ipython notebook, I got a "cannot import name accumulators" error.</p>
  
  <p>After some searching, I was thinking this had something to do with the install path for pyspark not being included in my Python Path, so I deleted my cluster and wanted to create a new one, using jupyter.sh, as initialization script.</p>
  
  <p>However, now my cluster wont startup at all, I get an error.
  Checking the log "dataproc-initialization-script-0_output" it simply says:</p>
  
  <p>/usr/bin/env: bash
  : No such file or directory</p>
  
  <p>Any ideas on what I'm missing here?</p>
  
  <hr>
  
  <p><strong>Edit:</strong></p>
  
  <p>I got the cluster to start with the public initialization script in gs://dataproc-initialization-actions/jupyter/jupyter.sh</p>
  
  <p>However, I'm still running into the same issues when trying to load pyspark in a pyspark notebook, 
  when I try something like "from pyspark import SparkConf"
  I get errors (weirdly enough, I get a different error if I run the same import a second time).</p>
  
  <p>Any feedback at this stage?</p>
  
  <p>This is the output of my notebook:
  <a href="http://i.stack.imgur.com/hnA03.png" rel="nofollow">html notebook output</a></p>