title: Hadoop pig join with time range
body: <p>I have two data sets</p>
  
  <pre><code>Master.txt
  ID,StartTime
  1,2013-04-01 00:01:37
  2,2013-04-01 00:01:37
  
  Transaction.txt
  ID,SurveyDate,Attr1
  1,2013-04-01 00:03:40,Success
  2,2013-05-01 00:01:30,Success
  </code></pre>
  
  <p>I want to join those two datasets with <code>ID</code> and (<code>SurveyDate-StartTime</code> &lt; 24hrs) and add <code>Attr1</code> to Master.</p>
  
  <p>I loaded Master and Transaction using Load in pig and I can do join on ID but I'm not able to find the Transactions which are within 24 hours of <code>Master.StartTime</code>.</p>
  
  <p><code>Master.txt</code> has significant number of records where Transactions are very few.</p>
  
  <p>here is code</p>
  
  <pre><code>master=load 'hdfs://localhost:9000/user/xyz/contact/master' using PigStorage(',') as (ID,StartTime) 
  
  transaction=load 'hdfs://localhost:9000/user/xyz/contact/transaction' using PigStorage(',') as 
  (ID,SurveyDate,Attr1)
  
  combine=join master by ID left outer, transaction ID
  </code></pre>
  
  <p>This is so far I have, as i understand from documentation conditions are not allowed when joining datasets. So not sure how would i add something like Master.StartTime-Transaction.SurveyDate &lt;=24hrs </p>