title: findspark.init() failing - Cannot get SPARK_HOME environment variables set correctly
body: <p>I'm new to using Spark and I'm attempting play with Spark on my local (windows) machine using Jupyter Notebook</p>
  
  <p>I've been following several tutorials for setting environment variables, as well as using multiple functions to do so via Python and cmd, and I cannot get any introductory PySpark code to work.</p>
  
  <p>When running (in Jupyter Notebook, using Python)</p>
  
  <pre class="lang-py prettyprint-override"><code>from pyspark import SparkContext
  from pyspark.sql import SQLContext
  sc = SparkContext('lcoal', 'Spark SQL') 
  </code></pre>
  
  <p>OR </p>
  
  <pre class="lang-py prettyprint-override"><code>from pyspark import SparkContext
  from pyspark.sql import SQLContext
  sc = SparkContext('C:\spark\spark-2.4.3-bin-hadoop2.7', 'Spark SQL') 
  </code></pre>
  
  <p>I get the error: </p>
  
  <pre class="lang-py prettyprint-override"><code>FileNotFoundError: [WinError 2] The system cannot find the file specified
  </code></pre>
  
  <p>Additionally, </p>
  
  <p>I attempted using findspark and run into the issue:</p>
  
  <pre class="lang-py prettyprint-override"><code>findspark.init()
  OR
  findspark.init("C:\spark\spark-2.4.3-bin-hadoop2.7")
  </code></pre>
  
  <p>I get the error:</p>
  
  <pre class="lang-py prettyprint-override"><code>IndexError: list index out of range
  </code></pre>
  
  <p>Which, from other posts around this topic I've been led to believe that the SPARK_HOME variable could be set incorrectly.</p>
  
  <p>My Environment variables are as follows: 
  My spark was extracted here:
  C:\spark\spark-2.4.3-bin-hadoop2.7</p>
  
  <p>HADOOP_HOME: C:\spark\spark-2.4.3-bin-hadoop2.7
  SPARK_HOME: C:\spark\spark-2.4.3-bin-hadoop2.7
  JAVA_HOME: C:\Program Files\Java\jdk1.8.0_201</p>
  
  <p>All of these including %SPARK_HOME%\bin have been added to my PATH variable.</p>
  
  <p>Lastly, When I cmd > cd %SPARK_HOME% it correctly brings me to the right directory, \spark\spark-2.4.3-bin-hadoop2.7</p>
  
  <p>As far as I can see, there are no issues with my environment variables so I'm unsure why pyspark through Juputer notebook cannot find my spark_home (or maybe that's not the issue).</p>
  
  <p>Would appreciate any and all help!</p>
  
  <p>Thanks!</p>