Compare two files and save the result into a pickle file
One suggestion is to use numpy to vectorize your dot product.

&gt;&gt;&gt; timeit.timeit('dot_product(range(10000),range(10000))', setup='from itertools import izip\ndef dot_product(v1, v2):\n    return sum(map(lambda x: x[0] * x[1], izip(v1, v2)))', number=1000)
2.666857957839966
&gt;&gt;&gt; timeit.timeit('np.dot(range(10000),range(10000))', setup='import numpy as np', number=1000)
0.9193508625030518


Another suggestion is to use multiple threads or processes to run multiple compare results at the same time. The libraries threading or multiprocessing would be useful here.