title: How to set the threshold for closest date when comparing two columns with dates
body: <p>I am extracting the rows from two csv files based on values in two columns. The name of columns in both files are same but values are different. I am matching the <code>'Education_Period'</code> column of df22 with <code>'Education_Period'</code> column of df33 to find the rows based on exactly same <code>'Education_Period'</code> and closest  to <code>'Program_startDate'</code>. For closest <code>'Program_startDate'</code>, the code compares the <code>'Program_startDate'</code> of df22 with <code>'Program_startDate'</code> of df33. If there are more than one rows with similar closest date , then it chooses one. At the end the output is written in df44 and df55 (as shown in output table below). Please note that df22 has less number of records than df33.</p>
  <p><strong>Here are two issues with below python code.</strong></p>
  <ol>
  <li>Since we are pairing the dataframes based on matching so at the end the number of records in df44 and df55 should be equal but I am getting different number in both (df44 has less than df55). How this can be fixed?</li>
  <li>Is there a way to set the threshold for closest <code>'Program_startDate'</code>. For example: when I am running this code on my huge datasets then I observed that for closest <code>'Program_startDate'</code> there are records where say 22/12/2009 is matched with 12/06/2015 as the closest date. I want the closest match upto 6 months or say any other number as a threshold. Please guide how I can do this in my below code?</li>
  </ol>
  <pre><code>Input:
  
  df2
  Student_IDs Education_Period  Waiting_Period     Program_startDate   
    
  23C                     100.5            5.5            29/03/2018
  34B                     77.2             3.0            12/12/2009
  11X                     77.2            8.5             14/09/2019
  88N                     99.9            12.0            20/03/2017
  22A                     77.2            12.0            30/03/2015
  
  df3
  Student_IDs Education_Period  Waiting_Period     Program_startDate   
   
  11X                     30.5             40.0         29/03/2018     
  99Y                     77.2             20.0        12/12/2009
  88Z                     14.1             19.1        14/09/2016
  12Z                     77.2             15.0        26/06/2018
  234M                    100.5            19.2        30/03/2015
  34M                     100.5            44.5        30/04/2018
  
  
  
  Output:
  
  df4
  Student_IDs Education_Period  Waiting_Period     Program_startDate   
    
  23C              100.5           5.5                     29/03/2018
  34B              77.2            3.0                     12/12/2009
  11X              77.2            8.5                     14/09/2019
  
  df5
  Student_IDs Education_Period  Waiting_Period     Program_startDate 
  234M             100.5           19.2                    30/04/2018
  99Y              77.2            20.0                    12/12/2009
  12Z              77.2            15.0                    26/06/2018    
  
  </code></pre>
  <pre><code>
  Python Code:
  df23 = df22.merge(df33, on='Education_Period', how='inner')
  df23['TD'] = (df23['Program_startDate_x']-df23['Program_startDate_y']).abs()
  df23 = df23.sort_values(by=['TD']).reset_index()
  df23i = df23.sort_values(by=['TD']).reset_index().drop_duplicates(subset=['Education_Period', 'Student_IDs_x']).groupby(['Education_Period'])['TD'].idxmin()
  df23i = [x[1] if type(x) == tuple else x for x in df23i]
  
  df23j = df23.sort_values(by=['TD']).reset_index().drop_duplicates(subset=['Education_Period', 'Student_IDs_y']).groupby(['Education_Period'])['TD'].idxmin()
  df23j = [x[1] if type(x) == tuple else x for x in df23j]
  df44 = df23.loc[df23i, ['Education_Period','Student_IDs_x', 'Waiting_Period_x', 'Program_startDate_x']].reset_index(drop=True).rename(columns={'Student_IDs_x':'Student_IDs', 'Waiting_Period_x':'Waiting_Period', 'Program_startDate_x':'Program_startDate'}).drop_duplicates(subset=['Education_Period', 'Student_IDs']).drop_duplicates(subset=['Education_Period', 'Student_IDs'])
  df55 = df23.loc[df23j, ['Education_Period','Student_IDs_y', 'Waiting_Period_y', 'Program_startDate_y']].reset_index(drop=True).rename(columns={'Student_IDs_y':'Student_IDs', 'Waiting_Period_y':'Waiting_Period', 'Program_startDate_y':'Program_startDate'}).drop_duplicates(subset=['Education_Period', 'Student_IDs']).drop_duplicates(subset=['Education_Period', 'Student_IDs'])
  
  
  </code></pre>