Python numpy memmap matrix multiplication
you might try to use np.memmap, and compute the 10x10 output matrix one element at a time.
so you just load the first row of the first matrix and the first column of the second, and then np.sum(row1 * col1).
Im trying to produce a usual matrix multiplication between two huge matrices (10*25,000,000).
My memory runs out when I do so. How could I use numpy's memmap to be able to handle this?
Is this even a good idea? I'm not so worried about the speed of the operation, I just want the result even if it means waiting some time. Thank you in advanced!
8 gbs ram, I7-2617M 1.5 1.5 ghz, Windows7 64 bits. Im using the 64 bit version of everything: python(2.7), numpy, scipy.
Edit1:
Maybe h5py is a better option?