title: FileNotFoundException in Azure Synapse when trying to delete a Parquet file and replace it with a dataframe created from the original file
body: <p>I am trying to delete an existing Parquet file and replace it with data in a dataframe that read the data in the original Parquet file before deleting it.  This is in Azure Synapse using PySpark.</p>
  <p>So I created the Parquet file from a dataframe and put it in the path:</p>
  <pre><code>full_file_path
  </code></pre>
  <p>I am trying to update this Parquet file.  From what I am reading, you can't edit a Parquet file so as a workaround, I am reading the file into a new dataframe:</p>
  <pre><code>df = spark.read.parquet(full_file_path)
  </code></pre>
  <p>I then create a new dataframe with the update:</p>
  <pre><code>df.createOrReplaceTempView(&quot;temp_table&quot;)
  df_variance = spark.sql(&quot;&quot;&quot;SELECT * FROM temp_table WHERE ....&quot;&quot;&quot;)
  </code></pre>
  <p>and the df_variance dataframe is created.</p>
  <p>I then delete the original file with:</p>
  <pre><code>mssparkutils.fs.rm(full_file_path, True)
  </code></pre>
  <p>and the original file is deleted.  But when I do any operation with the df_variance dataframe, like df_variance.count(), I get a FileNotFoundException error.  What I am really trying to do is:</p>
  <pre><code>df_variance.write.parquet(full_file_path)
  </code></pre>
  <p>and that is also a FileNotFoundException error.  But I am finding that any operation I try to do with the df_variance dataframe is producing this error.  So I am thinking it might have to do with the fact that the original full_file_path has been deleted and that the df_variance dataframe maintains some sort of reference to the (now deleted) file path, or something like that.  Please help.  Thanks.</p>