title: Merge multiple spark rows inside dataframe by ID into one row based on update_time
body: <p>we need to merge multiple rows based on ID into a single record using Pyspark. If there are multiple updates to the column, then we have to select the one with the last update made to it.
  Please note, NULL would mean there was no update made to the column in that instance.
  So, basically we have to create a single row with the consolidated updates made to the records.
  So,for example, if this is the dataframe ...</p>
  <p>Looking for similar answer, but in Pyspark .. <a href="https://stackoverflow.com/questions/50466270/merge-rows-in-a-spark-scala-dataframe">Merge rows in a spark scala Dataframe</a></p>
  <pre><code>------------------------------------------------------------
  | id       | column1          | column2         | updated_at |
  ------------------------------------------------------------
  | 123      | update1          | &lt;*no-update*&gt;   | 1634228709 |   
  | 123      | &lt;*no-update*&gt;    | 80              | 1634228724 |
  | 123      | update2          | &lt;*no-update*&gt;   | 1634229000 |
  </code></pre>
  <p>expected output is -</p>
  <pre><code>------------------------------------------------------------
  | id       | column1          | column2       | updated_at |
  ------------------------------------------------------------
  | 123      | update2          | 80            | 1634229000 |
  </code></pre>