title: ParseException: u"\nextraneous input '/' expecting {'SELECT', 'FROM', 'ADD',
body: <p>I am trying to read header files and corresponding <code>csv</code> from a directory, while writing csv's to <code>pyspark dataframe</code> throws error</p>
  
  <blockquote>
    <p>ParseException: u"\nextraneous input '/' expecting {'SELECT', 'FROM',
    'ADD',  IDENTIFIER, BACKQUOTED_IDENTIFIER}(line 1, pos 0)\n\n== SQL
    ==\n/persistent/4G/filtered_week1/cell_res\n^^^\n"</p>
  </blockquote>
  
  <p>My piece of code is attached here:</p>
  
  <p>Please let me know if I am doing something wrong here`</p>
  
  <pre><code>from pyspark.sql import SparkSession
  conf = spark.sparkContext._conf.setAll([('spark.executor.memory', '8g'), 
                                          ('spark.executor.cores', '2'), 
                                          ('spark.cores.max', '10'), 
                                          ('spark.driver.memory','8g')])
  spark = SparkSession.builder\
                      .config(conf=conf)\
                      .appName("Mergecsv mbnl")\
                      .getOrCreate()
  indir="/persistent/4G/filtered_week1/"
  #outfile = "/persistent/4G/week1_15Feb/Untitled Folder"
  csv_seperator=','
  header= glob.glob(indir + "/*.header")
  all_files = glob.glob(indir+ "/*.csv")
          for filename in all_files:
           for head in header:
              x= head.split('.sql')[0]
              if x in filename:
                  df1=spark.read.format("csv").option("header", "false").schema(x).load(filename)
                  df1.take(1)`
  </code></pre>