title: Pyspark: write to csv writes parquet instead of csv
body: <p>In the code below, out.csv is in parquet format. What option am I missing for it to write it as a csv file?</p>
  
  <pre><code>import py4j
  from pyspark import SparkConf, SparkContext
  from pyspark import HiveContext as hc
  import os
  from pyspark.sql import SQLContext, Row
  
  from datetime import datetime
  from pyspark.sql.types import DateType,StringType
  import pyspark.sql.functions as F
  
  os.environ['PYSPARK_SUBMIT_ARGS'] = '--packages com.databricks:spark-csv_2.11:1.5.0'
  conf = SparkConf().setMaster("local[64]").setAppName("My App")
  sc = SparkContext(conf = conf)
  sqlContext = SQLContext(sc)
  
  #read parquet file into DF
  df = sqlContext.read.parquet('/path/in_parquet')
  
  # Write to csv
  df_grouped = df.groupBy('column1').agg(F.sum('column2'))
  df_grouped.repartition(1).write.format("com.databricks.spark.csv").option("header", "true").save("/path/out.csv")
  </code></pre>
  
  <p><strong>Output:</strong></p>
  
  <p>the last few lines remaining in the console. also, here is the command i used to run the script:</p>
  
  <p>spark-submit --master local[*] --driver-memory 12g --packages com.databricks:spark-csv_2.11:1.2.0  MyPyspark.py</p>
  
  <pre><code>$ hdfs dfs -ls /path/out.csv
  Found 2 items
  -rw-r--r--   3 me devs          0 2017-06-29 12:16 /path/out.csv/_SUCCESS
  -rw-r--r--   3 me devs        552 2017-06-29 12:16 /path/out.csv/part-00000
  </code></pre>