6. The method of claim 4 , wherein the token similarity score for each matched token pair having alpha-numeric type tokens is determined by setting the similarity score to zero for the matched token pair when the tokens are not exact duplicates, and setting the similarity score for the matched token pair to one when the tokens are duplicates.
4. The method of claim 3 , wherein the token similarity score for each matched token pair is zero for a matched token pair if a first token and a second token of the matched token pair are of different types.