3. A system according to claim 2 , further comprising: a shingle module to divide each of the segments of the documents into sliding windows of N words; a window hasher to hash each of the sliding windows; a window comparison module to compare the hashed sliding windows for the original document with the hashed sliding windows for each of the remaining documents in the set; and a window similarity determination module to determine the similarity for each remaining document based on a number of the hashed sliding windows shared by the original document and that remaining document and based on a total number of the hashed sliding windows of the original document and that remaining document.
3. A system according to claim 2 , further comprising: a shingle module to divide each of the segments of the documents into sliding windows of N words; a window hasher to hash each of the sliding windows; a window comparison module to compare the hashed sliding windows for the original document with the hashed sliding windows for each of the remaining documents in the set; and a window similarity determination module to determine the similarity for each remaining document based on a number of the hashed sliding windows shared by the original document and that remaining document and based on a total number of the hashed sliding windows of the original document and that remaining document. 4. A system according to claim 3 , further comprising: a threshold module to apply a threshold to the similarity values assigned to the remaining documents; and the document designation module to designate the remaining documents with the similarity values that satisfy the threshold as near duplicates of the original document.