title: Pyspark : forward fill with last observation for a DataFrame
body: <p>Using Spark 1.5.1,</p>
  
  <p>I've been trying to forward fill <strong>null values</strong> with the last known observation for <strong>one column</strong> of my DataFrame.</p>
  
  <p>It is possible to start with a null value and for this case I would to backward fill this null value with the first knwn observation. However, If that too complicates the code, this point can be skipped.</p>
  
  <p>In this <a href="https://stackoverflow.com/questions/33621319/spark-scala-forward-fill-with-last-observation">post</a>, a solution in Scala was provided for a very similar problem by <a href="https://stackoverflow.com/users/1560062/zero323">zero323</a>.</p>
  
  <p>But, I don't know Scala and I don't succeed to ''translate'' it in Pyspark API code. It's possible to do it with Pyspark ? </p>
  
  <p>Thanks for your help.</p>
  
  <p>Below, a simple example sample input:</p>
  
  <pre><code>| cookie_ID     | Time       | User_ID   
  | ------------- | --------   |------------- 
  | 1             | 2015-12-01 | null 
  | 1             | 2015-12-02 | U1
  | 1             | 2015-12-03 | U1
  | 1             | 2015-12-04 | null   
  | 1             | 2015-12-05 | null     
  | 1             | 2015-12-06 | U2
  | 1             | 2015-12-07 | null
  | 1             | 2015-12-08 | U1
  | 1             | 2015-12-09 | null      
  | 2             | 2015-12-03 | null     
  | 2             | 2015-12-04 | U3
  | 2             | 2015-12-05 | null   
  | 2             | 2015-12-06 | U4
  </code></pre>
  
  <p>And the expected output:</p>
  
  <pre><code>| cookie_ID     | Time       | User_ID   
  | ------------- | --------   |------------- 
  | 1             | 2015-12-01 | U1
  | 1             | 2015-12-02 | U1
  | 1             | 2015-12-03 | U1
  | 1             | 2015-12-04 | U1
  | 1             | 2015-12-05 | U1
  | 1             | 2015-12-06 | U2
  | 1             | 2015-12-07 | U2
  | 1             | 2015-12-08 | U1
  | 1             | 2015-12-09 | U1
  | 2             | 2015-12-03 | U3
  | 2             | 2015-12-04 | U3
  | 2             | 2015-12-05 | U3
  | 2             | 2015-12-06 | U4
  </code></pre>