How to determine(with some confidence) if two extremely long lists of numbers (terabytes) are similar? I have two very large files with numbers seperated by commas. What is most efficient/fastest way to say with some confidence that these two files have same numbers. 

The main rule if if two distributions are the same, also the order of data does matter for example:
A = {1 2 4 5 6}
B = {6 5 4 2 1}
C = {0.5 0.5 1 2 3 4 5 6}

in above case A and C are still similar but A and B are totally different

To be more clear lets split the question into 3 main tasks


Given large lists A,B find if lets say 50% of A is contained in B (Most efficient technique without comparing all numbers from list)
Given A,B match above condition check that the overlapping numbers have similar order as mentioned above i.e. if 2 in A is followed by 1 same should happen in B. The positions of 1,2 can be different in B.
Select two independent batches from A and B and test if those are from Poission distribution, binomial distribution.
Check out Jaccard Similarity, as it gives a measure of set similarity.  See MinHash, locality sensitive hashing.