title: If you save a DataFrame to disk, will Spark load that data if you use that DataFrame lower in the script?
body: <p>If you load some data, compute a DataFrame, write that to disk and then use the DataFrame later... assuming it isn't still cached in RAM (lets say there wasn't enough), would Spark be smart enough to load the data from disk rather than recompute the DataFrame from the original data?</p>
  
  <p>For example:</p>
  
  <pre><code>df1 = spark.read.parquet('data/df1.parquet')
  df2 = spark.read.parquet('data/df2.parquet')
  
  joined = df1.join(df2, df1.id == df2.id)
  joined.write.parquet('data/joined.parquet')
  
  computed = joined.select('id').withColummn('double_total', 2 * joined.total)
  computed.write.parquet('data/computed.parquet')
  </code></pre>
  
  <p>Under the right circumstances, when we store <code>computed</code>, will it load the <code>joined</code> DataFrame from <code>data/joined.parquet</code> or will it always re-compute by loading/joining <code>df1</code>/<code>df2</code> if it isn't currently caching <code>joined</code>?</p>