title: Find date range overlap in python
body: <p>I am trying to find a more efficient way of finding overlapping data ranges (start/end dates provided per row) in a dataframe based on a specific column (id). Dataframe is sorted on the 'from' column. I think there is a way to avoid the double <code>apply</code> function as I did:</p>
  <pre class="lang-python prettyprint-override"><code>import pandas as pd
  from datetime import datetime
  
  df = pd.DataFrame(columns=['id','from','to'], index=range(5), \
                    data=[[878,'2006-01-01','2007-10-01'],
                          [878,'2007-10-02','2008-12-01'],
                          [878,'2008-12-02','2010-04-03'],
                          [879,'2010-04-04','2199-05-11'],
                          [879,'2016-05-12','2199-12-31']])
  
  df['from'] = pd.to_datetime(df['from'])
  df['to'] = pd.to_datetime(df['to'])
  
  
      id  from        to
  0   878 2006-01-01  2007-10-01
  1   878 2007-10-02  2008-12-01
  2   878 2008-12-02  2010-04-03
  3   879 2010-04-04  2199-05-11
  4   879 2016-05-12  2199-12-31
  </code></pre>
  <p>I used the &quot;apply&quot; function to loop on all groups and within each group, I use &quot;apply&quot; per row:</p>
  <pre class="lang-python prettyprint-override"><code>def check_date_by_id(df):
      
      df['prevFrom'] = df['from'].shift()
      df['prevTo'] = df['to'].shift()
      
      def check_date_by_row(x):
          
          if pd.isnull(x.prevFrom) or pd.isnull(x.prevTo):
              x['overlap'] = False
              return x
          
          latest_start = max(x['from'], x.prevFrom)
          earliest_end = min(x['to'], x.prevTo)
          x['overlap'] = int((earliest_end - latest_start).days) + 1 &gt; 0
          return x
      
      return df.apply(check_date_by_row, axis=1).drop(['prevFrom','prevTo'], axis=1)
  
  df.groupby('id').apply(check_date_by_id)
  
      id  from        to          overlap
  0   878 2006-01-01  2007-10-01  False
  1   878 2007-10-02  2008-12-01  False
  2   878 2008-12-02  2010-04-03  False
  3   879 2010-04-04  2199-05-11  False
  4   879 2016-05-12  2199-12-31  True
  </code></pre>
  <p>My code was inspired from the following links :</p>
  <ul>
  <li><p><a href="https://stackoverflow.com/questions/9044084/efficient-date-range-overlap-calculation-in-python">Efficient date range overlap calculation in python?</a></p>
  </li>
  <li><p><a href="https://stackoverflow.com/questions/6821156/how-to-find-range-overlap-in-python">How to find range overlap in python?</a></p>
  </li>
  </ul>