title: How to calculate cumulative sum using sqlContext
body: <p>I know we can use <a href="https://databricks.com/blog/2015/07/15/introducing-window-functions-in-spark-sql.html" rel="nofollow">Window function in pyspark</a> to calculate cumulative sum. But Window is only supported in HiveContext and not in SQLContext. I need to use SQLContext as HiveContext cannot be run in multi processes.</p>
  
  <p>Is there any efficient way to calculate cumulative sum using SQLContext? A simple way is to load the data into the driver's memory and use numpy.cumsum, but the con is the data need to be able to fit into the memory</p>