title: save updates into dataframe and reuse the saved dataframe in spark scala
body: <p>I get multiple incoming files and i have to compare each incoming file with the source file then merge and replace the old rows with the new rows and append the extra rows if any present in the source file. Afterwords I have to use the updated sourcefile and compare with another incoming file, update it and so the process goes on. </p>
  
  <p>I have so far created the dataframe for each file and compared and merged using join. i want to save all the updates done in the source file and use the updated source file again to compare and update incomming files.</p>
  
  <pre><code>          val merge = df.union(dfSource.join(df, Seq( "EmployeeID" ),joinType= "left_anti").orderBy("EmployeeID") )
  
            merge.write.mode ("append").format("text").insertInto("dfSource")
                merge.show()
  </code></pre>
  
  <p>I tried this way but it dosent update my dfSource dataframe. could somebody help please.</p>
  
  <p>Thanks </p>