title: Parallelism of window-functions in Spark 2
body: <p>I have a rather simple (but essential) question about Spark window functions (such as e.g. <code>lead, lag, count, sum, row_number</code> etc):</p>
  
  <p>If a specify my window as <code>Window.partitionBy(lit(0))</code> (i.e. I need to run the window fucntion accross the entire dataframe), is the window aggregate function running in parallel, or are all records moved in one single tasks?</p>
  
  <p>EDIT:</p>
  
  <p>Especially for "rolling" operations (e.g. rolling average using something like <code>avg(...).Window.partitionBy(lit(0)).orderBy(...).rowsBetween(-10,10)</code>), this operation could very well be split up into different tasks even tough all data is in the same partition of the window because only 20 row are needed at once to compute the average</p>