title: Dataframe save after join is creating numerous part files
body: <p>I am trying to learn programming with Dataframes. With below code i am trying to join two CSV on a column and then saving it as a combined CSV. Running this code in SCALA IDE i am seeing almost 200 small part files as a result. Could you please help me understand what is going wrong here-</p>
  
  <pre><code>import org.apache.spark.SparkContext
  
  object JoinData {  
      def main(args: Array[String]) {
      val sc = new SparkContext(args(0), "Csv Joining example")    
      val sqlContext = new org.apache.spark.sql.SQLContext(sc)  
      val df1 = sqlContext.load("com.databricks.spark.csv", Map("path" -&gt; args(1), "header" -&gt; "true"))
      val df2 = sqlContext.load("com.databricks.spark.csv", Map("path" -&gt; args(2), "header" -&gt; "true")) 
      import org.apache.spark.sql.functions._
      val df_join = df1.join(df2, df1("Dept") === df2("dept"), "inner") 
      df_join.repartition(1) //This is also not helping
      //Below line is generating 200 part files in Output_join folder
      df_join.save("Output_join","com.databricks.spark.csv",  org.apache.spark.sql.SaveMode.Overwrite)
  
       }
  }
  </code></pre>
  
  <p><strong>Program arguments used -</strong>
      local src/main/resources/emp.csv src/main/resources/dept.csv</p>
  
  <p><strong>CSV data being used</strong></p>
  
  <pre><code>empId,empName,Dept,salution
  111,ABC,sales,mr
  112,ABC,it,mr
  113,ABC,tech,mr
  114,ABC,sales,mr
  115,ABC,sales,mr
  116,ABC,it,mr
  117,ABC,tech,mr
  
  dept,name
  sales,Sales of Cap
  it,Internal Training
  tech,Tech staff
  support,support services
  </code></pre>
  
  <p><strong>Console Output</strong></p>
  
  <pre><code>[Stage 4:&gt;                                                        (2 + 1) / 200]
  [Stage 4:=&gt;                                                       (4 + 1) / 200]
  [Stage 4:=&gt;                                                       (6 + 1) / 200]
  [Stage 4:==&gt;                                                      (8 + 1) / 200]
  [Stage 4:===&gt;                                                    (11 + 1) / 200]
  [Stage 4:===&gt;                                                    (14 + 1) / 200]
  [Stage 4:====&gt;                                                   (17 + 1) / 200]
  [Stage 4:=====&gt;                                                  (19 + 1) / 200]
  [Stage 4:=====&gt;                                                  (21 + 1) / 200]
  [Stage 4:======&gt;                                                 (24 + 1) / 200]
  [Stage 4:=======&gt;                                                (26 + 1) / 200]
  [Stage 4:=======&gt;                                                (28 + 1) / 200]
  [Stage 4:========&gt;                                               (30 + 1) / 200]
  [Stage 4:========&gt;                                               (32 + 1) / 200]
  [Stage 4:=========&gt;                                              (34 + 1) / 200]
  [Stage 4:==========&gt;                                             (37 + 1) / 200]
  [Stage 4:===========&gt;                                            (40 + 1) / 200]
  [Stage 4:============&gt;                                           (43 + 1) / 200]
  [Stage 4:============&gt;                                           (46 + 1) / 200]
  [Stage 4:=============&gt;                                          (49 + 1) / 200]
  [Stage 4:==============&gt;                                         (52 + 1) / 200]
  [Stage 4:===============&gt;                                        (55 + 1) / 200]
  [Stage 4:================&gt;                                       (58 + 1) / 200]
  [Stage 4:=================&gt;                                      (61 + 1) / 200]
  [Stage 4:=================&gt;                                      (64 + 1) / 200]
  [Stage 4:==================&gt;                                     (67 + 1) / 200]
  [Stage 4:===================&gt;                                    (69 + 1) / 200]
  [Stage 4:====================&gt;                                   (72 + 1) / 200]
  [Stage 4:=====================&gt;                                  (75 + 1) / 200]
  [Stage 4:=====================&gt;                                  (78 + 1) / 200]
  [Stage 4:======================&gt;                                 (81 + 1) / 200]
  [Stage 4:=======================&gt;                                (84 + 1) / 200]
  [Stage 4:========================&gt;                               (87 + 1) / 200]
  [Stage 4:=========================&gt;                              (90 + 1) / 200]
  [Stage 4:=========================&gt;                              (92 + 1) / 200]
  [Stage 4:==========================&gt;                             (95 + 1) / 200]
  [Stage 4:===========================&gt;                            (98 + 1) / 200]
  [Stage 4:===========================&gt;                           (101 + 1) / 200]
  [Stage 4:============================&gt;                          (104 + 1) / 200]
  [Stage 4:=============================&gt;                         (107 + 1) / 200]
  [Stage 4:==============================&gt;                        (110 + 1) / 200]
  [Stage 4:===============================&gt;                       (113 + 1) / 200]
  [Stage 4:===============================&gt;                       (116 + 1) / 200]
  [Stage 4:================================&gt;                      (119 + 1) / 200]
  [Stage 4:=================================&gt;                     (122 + 1) / 200]
  [Stage 4:=================================&gt;                     (123 + 1) / 200]
  [Stage 4:==================================&gt;                    (126 + 1) / 200]
  [Stage 4:===================================&gt;                   (129 + 1) / 200]
  [Stage 4:====================================&gt;                  (132 + 1) / 200]
  [Stage 4:=====================================&gt;                 (135 + 1) / 200]
  [Stage 4:=====================================&gt;                 (138 + 1) / 200]
  [Stage 4:======================================&gt;                (140 + 1) / 200]
  [Stage 4:======================================&gt;                (141 + 1) / 200]
  [Stage 4:=======================================&gt;               (144 + 1) / 200]
  [Stage 4:========================================&gt;              (148 + 1) / 200]
  [Stage 4:=========================================&gt;             (151 + 1) / 200]
  [Stage 4:==========================================&gt;            (154 + 1) / 200]
  [Stage 4:==========================================&gt;            (156 + 2) / 200]
  [Stage 4:===========================================&gt;           (159 + 1) / 200]
  [Stage 4:============================================&gt;          (161 + 1) / 200]
  [Stage 4:============================================&gt;          (162 + 1) / 200]
  [Stage 4:=============================================&gt;         (164 + 1) / 200]
  [Stage 4:=============================================&gt;         (165 + 1) / 200]
  [Stage 4:==============================================&gt;        (168 + 1) / 200]
  [Stage 4:===============================================&gt;       (171 + 1) / 200]
  [Stage 4:===============================================&gt;       (174 + 1) / 200]
  [Stage 4:================================================&gt;      (177 + 1) / 200]
  [Stage 4:=================================================&gt;     (180 + 1) / 200]
  [Stage 4:==================================================&gt;    (183 + 1) / 200]
  [Stage 4:===================================================&gt;   (186 + 1) / 200]
  [Stage 4:===================================================&gt;   (189 + 1) / 200]
  [Stage 4:=====================================================&gt; (193 + 1) / 200]
  [Stage 4:=====================================================&gt; (196 + 1) / 200]
  [Stage 4:======================================================&gt;(199 + 1) / 200]
  </code></pre>