title: How to use PySpark to get certain durations?
body: <p>Suppose we have already defined the DataFrame below in PySpark. And, how to use PySpark to get the duration (in mins) between the first biking event and the driving event (till the next driving event)?  (Save the results into dataframe, including start_time, start_event, end_time, end_event, and durations, etc)</p>
  <p>Notes: there can be other events between biking and driving, such as a person can start from biking, running then driving.</p>
  <p>For example, the duration between the first biking event and the coming driving event is:</p>
  <p>Start time: the Event should be 'biking' and Action should be 'start'</p>
  <p>End time: the Event should be 'driving' and Action should be 'end'</p>
  <p>so the duration in date '3/01/2018' is :   8:12 - 5:12 = 3 hours = 180 mins</p>
  <p>('start_time' -&gt; timestamp happen at: 3/1/2018 5:12  &amp; 'end_time' -&gt; timestamp happen at: 3/1/2018 8:12)</p>
  <p>Notes: there can be multiple biking events then driving events coming, only need to record the first biking event till the next driving event.</p>
  <p>the duration for date '3/02/2018' is:  14:12 (driving end as the end time) - 9:12 (first biking start as the start time) = 14:12 - 9:12 = 5 hours = 300 mins</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th></th>
  <th>TimeDetails</th>
  <th>Event</th>
  <th>Action</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>1</td>
  <td>3/1/2018 5:12</td>
  <td>Biking</td>
  <td>start</td>
  </tr>
  <tr>
  <td>2</td>
  <td>3/1/2018 6:12</td>
  <td>Biking</td>
  <td>end</td>
  </tr>
  <tr>
  <td>3</td>
  <td>3/1/2018 7:12</td>
  <td>Driving</td>
  <td>start</td>
  </tr>
  <tr>
  <td>4</td>
  <td>3/1/2018 8:12</td>
  <td>Driving</td>
  <td>end</td>
  </tr>
  <tr>
  <td>5</td>
  <td>3/2/2018 9:12</td>
  <td>Biking</td>
  <td>start</td>
  </tr>
  <tr>
  <td>6</td>
  <td>3/2/2018 10:12</td>
  <td>Biking</td>
  <td>end</td>
  </tr>
  <tr>
  <td>7</td>
  <td>3/2/2018 11:12</td>
  <td>Biking</td>
  <td>start</td>
  </tr>
  <tr>
  <td>8</td>
  <td>3/2/2018 12:12</td>
  <td>Biking</td>
  <td>end</td>
  </tr>
  <tr>
  <td>9</td>
  <td>3/2/2018 13:12</td>
  <td>Driving</td>
  <td>start</td>
  </tr>
  <tr>
  <td>9</td>
  <td>3/2/2018 14:12</td>
  <td>Driving</td>
  <td>end</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>Below is some of my code:</p>
  <pre><code>biking_df = df.filter(df.Event == 'Biking' &amp; (df.Action == 'start'))
  driving_df = df.filter(df.Event == 'Driving' &amp; (df.Action == 'end'))
  </code></pre>
  <p>Can someone please provide me with some code in PySpark?
  Thanks a lot</p>