Data duplication optimization
If you are looking for only for identical copies for files, then you can calculate a hash like md5sum on each file and compare that, which is much faster than analysing the content.

If you’re looking for a similarity metric then I think you should be able to use a faster algorithm than the one that SequenceMatcher uses, which finds the longest exactly matching sequence, which takes quadratic time on average. I suggest you look at other similarity metrics, such as Levenshtein or Hamming distance, which will be faster to compute.