5. A method of identifying duplicate records between a first database and a second database, each database comprised of a plurality of records arranged in rows and columns comprising: assigning a unique identifier to all records in the first database and second database that do not already have a unique identifier; creating a blocking subset of between 1 and all of the columns in the second database; mapping corresponding columns in the first database onto the blocking subset; creating a set (S) of subsets (s), the subsets (s) consisting of the unique identifiers of records (r) from the first database and/or second database wherein the number of subsets (s) is less than or equal to a heuristic value wherein the heuristic value is a positive integer, m; and each record (r) in subset (s) has the same value in at least one of the columns in the blocking subset, the creating a set (S) of subsets (s) step constructs set (S) by first constructing a set (T) which may contain both sets with more than m members and sets with fewer than m members; for every subset (s) in set (S), applying a pair-wise matching algorithm to compare every record (r) in the subset (s) to one another, optionally omitting comparisons of records (r) which were already compared in other subsets (s); and outputting the unique identifiers of record matches identified by the pair-wise matching algorithm, optionally omitting duplicate matches.
10. The method of claim 5 wherein the outputting step further comprises omitting duplicate matches.