title: Overwriting the parquet file throws exception in spark
body: <p>I am trying to read the parquet file from hdfs location, do some transformations and overwrite the file in the same location. I had to overwrite the file in the same location because I had to run the same code multiple times. </p>
  
  <p>Here is the code I have written</p>
  
  <pre><code>val df = spark.read.option("header", "true").option("inferSchema", "true").parquet("hdfs://master:8020/persist/local/")
  //after applying some transformations lets say the final dataframe is transDF which I want to overwrite at the same location.
  transDF.write.mode("overwrite").parquet("hdfs://master:8020/persist/local/")
  </code></pre>
  
  <p>Now the problem is before reading the parquet file from the given location, spark for some reason I believe it deletes the file at the given location because of overwrite mode. So when executing the code I get the following error.</p>
  
  <pre><code>File does not exist: hdfs://master:8020/persist/local/part-00000-e73c4dfd-d008-4007-8274-d445bdea3fc8-c000.snappy.parquet
  </code></pre>
  
  <p>Any suggestions on how to solve this problem? Thanks. </p>