28. A method for determining a likelihood of two documents describing similar subject matter, the method comprising: obtaining a set of tokens for each of a first and a second document, wherein each set of tokens represents a series of characters found in its corresponding document; obtaining a set of matched token pairs, each matched token pair comprising a first token from the set of tokens corresponding to the first document and a second token from the set of tokens corresponding to the second document; wherein the second token matches the first token; wherein each matched token pair has a term frequency—inverse document frequency (TFIDF) score and an edit distance score; wherein the TFIDF score represents an average of the number of times each token of the matched token pair appears in its respective document times a harmonic mean of an inverse document frequency in a document corpus of each token of the matched token pair; and wherein the edit distance score represents a relative similarity of the matched token pair; determining a similarity score for the first and second documents according to the token similarity scores of the token pairs in the set of matched token pairs by dividing a weighted common similarity score by the sum of a matched tokens TFIDF score and an unmatched tokens TFIDF score, wherein: the weighted common similarity score comprises a sum of a product of each TFIDF score and each edit distance score for the token pairs in the set of matched token pairs; the matched tokens TFIDF score comprises a sum of each TFIDF score for the token pairs in the set of matched token pairs; and the unmatched tokens TFIDF score comprises a sum of each TFIDF score for each unmatched token of the first document; and providing the determined similarity score as the likelihood of the first and second documents describing similar subject matter.
29. The method of claim 28 , wherein the unmatched tokens TFIDF score comprises a sum of each TFIDF score for each unmatched token of the first document plus a sum of each TFIDF score for each unmatched token of the second document.