title: Calculate time difference between consecutive rows in pairs per group in pyspark
body: <p>I want to calculate the time spent per SeqID for each user. I have a dataframe like this.
  However, the time is split between two actions for every user, <code>Action_A and Action_B.</code>
  The total time per user, per seqID would be sum across all such pairs</p>
  
  <p>For first user, it is <code>5 + 3 [(2019-12-10 10:00:00 - 2019-12-10 10:05:00) + (2019-12-10 10:20:00 - 2019-12-10 10:23:00)]
  </code></p>
  
  <p>So first user has ideally spent <code>8 mins</code> for SeqID 1 (and not <code>23 mins</code>). </p>
  
  <p>Similarly user 2 has spent <code>1 + 5 = 6 mins</code></p>
  
  <p>How can I calculate this using pyspark?</p>
  
  <pre><code>data = [(("ID1", 15, "2019-12-10 10:00:00", "Action_A")), 
          (("ID1", 15, "2019-12-10 10:05:00", "Action_B")),
          (("ID1", 15, "2019-12-10 10:20:00", "Action_A")),
          (("ID1", 15, "2019-12-10 10:23:00", "Action_B")),
          (("ID2", 23, "2019-12-10 11:10:00", "Action_A")),
          (("ID2", 23, "2019-12-10 11:11:00", "Action_B")),
          (("ID2", 23, "2019-12-10 11:30:00", "Action_A")),
          (("ID2", 23, "2019-12-10 11:35:00", "Action_B"))]
  df = spark.createDataFrame(data, ["ID", "SeqID", "Timestamp", "Action"])
  df.show()
  
  +---+-----+-------------------+--------+
  | ID|SeqID|          Timestamp|  Action|
  +---+-----+-------------------+--------+
  |ID1|   15|2019-12-10 10:00:00|Action_A|
  |ID1|   15|2019-12-10 10:05:00|Action_B|
  |ID1|   15|2019-12-10 10:20:00|Action_A|
  |ID1|   15|2019-12-10 10:23:00|Action_B|
  |ID2|   23|2019-12-10 11:10:00|Action_A|
  |ID2|   23|2019-12-10 11:11:00|Action_B|
  |ID2|   23|2019-12-10 11:30:00|Action_A|
  |ID2|   23|2019-12-10 11:35:00|Action_B|
  +---+-----+-------------------+--------+
  </code></pre>
  
  <p>Once I have the data for each pair, I can sum across the group (ID, SeqID)</p>
  
  <p>Expected output (could be seconds also)</p>
  
  <pre><code>+---+-----+--------+
  | ID|SeqID|Dur_Mins|
  +---+-----+--------+
  |ID1|   15|       8|
  |ID2|   23|       6|
  +---+-----+--------+
  </code></pre>