title: DataFrame Definintion is lazy evaluation
body: <p>I am new to spark and learning it. can someone help with below question</p>
  
  <p>The quote in spark definitive regarding dataframe definition is "In general, Spark will fail only at job execution time rather than DataFrame definition time—even if,
  for example, we point to a file that does not exist. This is due to lazy evaluation,"</p>
  
  <p>so I guess <code>spark.read.format().load()</code> is dataframe definition. On top of this created dataframe we apply transformations and action and load is read API and not transformation if I am not wrong. </p>
  
  <p>I tried to "file that does not exist" in load and I am thinking this is dataframe definition. but I got below error. according to the book it should not fail right?. I am surely missing something. can someone help on this?</p>
  
  <pre><code>df=spark.read.format('csv')
  .option('header', 
  'true').option('inferschema', 'true')
  .load('/spark_df_data/Spark-The-Definitive-Guide/data/retail-data/by-day/2011-12-19.csv')
  </code></pre>
  
  <p><strong>Error</strong> </p>
  
  <pre><code>Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
    File "/usr/hdp/current/spark2-client/python/pyspark/sql/readwriter.py", line 166, in load
      return self._df(self._jreader.load(path))
    File "/usr/hdp/current/spark2-client/python/lib/py4j-0.10.6-src.zip/py4j/java_gateway.py", line 1160, in __call__
    File "/usr/hdp/current/spark2-client/python/pyspark/sql/utils.py", line 69, in deco
      raise AnalysisException(s.split(': ', 1)[1], stackTrace)
  pyspark.sql.utils.AnalysisException: u'Path does not exist: /spark_df_data/Spark-The-Definitive-Guide/data/retail-data/by-day/2011-12-19.csv;' 
  </code></pre>
  
  <p>why dataframe definition is referring Hadoop metadata when it is lazy evaluated?</p>