title: Date difference between consecutive rows - Pyspark Dataframe
body: <p>I have a table with following structure</p>
  
  <pre><code>USER_ID     Tweet_ID                 Date
    1           1001       Thu Aug 05 19:11:39 +0000 2010
    1           6022       Mon Aug 09 17:51:19 +0000 2010
    1           1041       Sun Aug 19 11:10:09 +0000 2010
    2           9483       Mon Jan 11 10:51:23 +0000 2012
    2           4532       Fri May 21 11:11:11 +0000 2012
    3           4374       Sat Jul 10 03:21:23 +0000 2013
    3           4334       Sun Jul 11 04:53:13 +0000 2013
  </code></pre>
  
  <p>Basically what I would like to do is have a <strong>PysparkSQL</strong> query that calculates the date difference (in seconds) for consecutive records with the same user_id number. The expected result would be:</p>
  
  <pre><code>1      Sun Aug 19 11:10:09 +0000 2010 - Mon Aug 09 17:51:19 +0000 2010     839930
  1      Mon Aug 09 17:51:19 +0000 2010 - Thu Aug 05 19:11:39 +0000 2010     340780
  2      Fri May 21 11:11:11 +0000 2012 - Mon Jan 11 10:51:23 +0000 2012     1813212
  3      Sun Jul 11 04:53:13 +0000 2013 - Sat Jul 10 03:21:23 +0000 2013     5510
  </code></pre>