title: pyspark - For each key value, how to get first and last notNull value (based on timestamp) for all other columns
body: <p>I have a pyspark dataframe with values like this. I want to get first/last notNull value of state and their corresponding timestamps in 4 separate columns. Do similarly for country column also. Output result will have total 9 columns as mentioned in the bottom. Please help me how to do this in (pyspark or SQL) and fast performance. I have a HUGE table with millions of rows and 10 columns i need to this first/last and their timestamps</p>
  <pre><code>empId   timestamp      stateID  countryID
  1   5/1/2022 10:10am    CA  
  1                       CA        USA
  1   5/2/2022 11:11pm    CT        USA
  1                       NJ        USA
  1   5/10/2022 12:12pm             UK
  2                       VA        USA
  2   5/9/2022 12:15am    TX  
  2   5/10/2022 09:09am   CA        USA
  3                       NY        USA
  3   5/16/2022 09:15pm   MO        Japan
  3   5/17/2022 04:04am   AL        USA
  3   5/20/2022 07:07pm             UK
  </code></pre>
  <p><img src="https://i.stack.imgur.com/o0XfO.png" alt="enter image description here" /></p>