title: Pyspark : Interpolation of missing values in pyspark dataframe observed
body: <p>I am trying to clean a time series dataset using spark that is not fully populated and fairly large.  </p>
  
  <p>What I would like to do is convert the following dataset as such</p>
  
  <pre><code>Group | TS          |  Value
  ____________________________
  A     | 01-01-2018  |  1
  A     | 01-02-2018  |  2
  A     | 01-03-2018  |  
  A     | 01-04-2018  |  
  A     | 01-05-2018  |  5
  A     | 01-06-2018  |  
  A     | 01-07-2018  |  10
  A     | 01-08-2018  |  11
  </code></pre>
  
  <p>and convert it to the following</p>
  
  <pre><code>Group | TS          |  Value&gt;
  ____________________________
  A     | 01-01-2018  |  1
  A     | 01-02-2018  |  2
  A     | 01-03-2018  |  3
  A     | 01-04-2018  |  4
  A     | 01-05-2018  |  5
  A     | 01-06-2018  |  7.5
  A     | 01-07-2018  |  10
  A     | 01-08-2018  |  11
  </code></pre>
  
  <p>If you can help that would be greatly appreciated.</p>