title: Merge multiple csv files to one csv file in Azure Blob Storage using pyspark
body: <p>I am using below code to save the csv files back to blob storage, though it is creating multiple files as it runs in loop. Now I would like to merge them into one single csv file. Though I have used dbutils.fs.cp/mv, it is not helpful</p>
  
  <pre><code>while start_date &lt;= end_date:
  df = spark.read.format("com.databricks.spark.csv").options(header="true", inferschema="true").load(inputFilePath)
  df.coalesce(1).write.mode("append").option("header","true").format("com.databricks.s`park.csv").save(TargetPath)`
  </code></pre>
  
  <p>A similar request has been posted below, but it has been done using pandas data frame and I am looking something with spark dataframe.
  "<a href="https://stackoverflow.com/questions/56767724/copy-data-from-multiple-csv-files-into-one-csv-file">Copy data from multiple csv files into one csv file</a>"</p>