title: Parallelized record combining - matching on multiple keys
body: <p>I have been looking at using MapReduce to build a parallelized record combining system. The language doesn't matter, I can use a pre-existing library such as Hadoop or build my own if necessary, I'm not worried about that.</p>
  
  <p>The problem that I keep running into, however, is that I need the records to be matched on multiple criteria. For example: I may need to match the records based on person's name <strong>or</strong> the person's phone number, but not necessarily the person's name <strong>and</strong> phone number.</p>
  
  <p>For instance, given the following keys for each record:</p>
  
  <ol>
  <li>'John Smith' and '555-555-5555'</li>
  <li>'Jane Smith' and '555-555-5555'</li>
  <li>'John Smith' and '555-555-1111'</li>
  </ol>
  
  <p>I want the system to take all three records, figure out that they match on one of the keys, and combine them into a single combined record that has both names ('John Smith' and 'Jane Smith') as well as both phone numbers ('555-555-5555' and '555-555-1111').</p>
  
  <p>Is this something that I can accomplish using MapReduce? If so, how would I go about matching the keys produced by the Map function so that all of the matched records can be passed into the Reduce function.* Alternatively,  is there a different/better way I could be doing this? My only real requirement is that I need it parallelized.</p>
  
  <p>[*] Please note: I am assuming that the Reduce function could be used in such a way that each call to the Reduce function produces a single combined record, rather than the Reduce function producing a single result for the entire job.</p>