title: Calculate value based on value from same column of the previous row in spark
body: <p>I have an issue where I have to calculate a column using a formula that uses the value from the calculation done in the previous row.</p>
  
  <p>I am unable to figure it out using <code>withColumn</code> API.</p>
  
  <p>I need to calculate a new column, using the formula:</p>
  
  <p><code>MovingRate = MonthlyRate + (0.7 * MovingRatePrevious)</code></p>
  
  <p>... where the <code>MovingRatePrevious</code> is the <code>MovingRate</code> of the prior row. </p>
  
  <p>For month 1, I have the value so I do not need to re-calculate that but I need that value to be able to calculate the subsequent rows. I need to partition by Type.</p>
  
  <p>This is my original dataset:</p>
  
  <p><a href="https://i.stack.imgur.com/PZYKQ.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/PZYKQ.jpg" alt="enter image description here"></a></p>
  
  <p>Desired results in MovingRate column:</p>
  
  <p><a href="https://i.stack.imgur.com/AeLHN.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/AeLHN.jpg" alt="enter image description here"></a></p>