How to determine(with some confidence) if two extremely long lists of numbers (terabytes) are similar?
Check out Jaccard Similarity, as it gives a measure of set similarity.  See MinHash, locality sensitive hashing.