title: Does Spark distributes dataframe across nodes internally?
body: <p>I am trying to use Spark for processing csv file on cluster. I want to understand if I need to explicitly read the file on each of the worker nodes to do the processing in parallel or will the driver node read the file and distribute the data across cluster for processing internally? (I am working with Spark 2.3.2 and Python)</p>
  
  <p><strong>I know RDD's can be parallelized using SparkContext.parallelize() but what in case of Spark DataFrames?</strong></p>
  
  <pre><code>if __name__=="__main__":
       spark=SparkSession.builder.appName('myApp').getOrCreate()
       df=spark.read.csv('dataFile.csv',header=True)
       df=df.filter("date&gt;'2010-12-01' AND date&lt;='2010-12-02' AND town=='Madrid'")
  </code></pre>
  
  <p><strong>So if I am running the above code on cluster, will the entire operation be done by driver node or will it distribute df across cluster and each worker perform processing on its data partition?</strong></p>