title: getting error name 'spark' is not defined
body: <p>This is the code i used:</p>
  
  <pre><code>df = None
  
  from pyspark.sql.functions import lit
  
  for category in file_list_filtered:
      data_files = os.listdir('HMP_Dataset/'+category)
  
      for data_file in data_files:
          print(data_file)
          temp_df = spark.read.option('header', 'false').option('delimiter', ' ').csv('HMP_Dataset/'+category+'/'+data_file, schema = schema)
          temp_df = temp_df.withColumn('class', lit(category))
          temp_df = temp_df.withColumn('source', lit(data_file))
  
          if df is None:
              df = temp_df
          else:
              df = df.union(temp_df)
  </code></pre>
  
  <p>and i got this error:</p>
  
  <pre><code>NameError                                 Traceback (most recent call last)
  &lt;ipython-input-4-4296b4e97942&gt; in &lt;module&gt;
        9     for data_file in data_files:
       10         print(data_file)
  ---&gt; 11         temp_df = spark.read.option('header', 'false').option('delimiter', ' ').csv('HMP_Dataset/'+category+'/'+data_file, schema = schema)
       12         temp_df = temp_df.withColumn('class', lit(category))
       13         temp_df = temp_df.withColumn('source', lit(data_file))
  
  NameError: name 'spark' is not defined
  </code></pre>
  
  <p>How can i solve it?</p>