Use hard drive instead of RAM in Python
If all you need is a virtualization of the disk as a large RAM memory you might set up a swap file on the system. The kernel will then automatically swap pages in and out as needed, using heuristics to figure out what pages should be swapped and which should stay on disk.
I'd like to know if there's a method or a Python Package that can make me use a large dataset without writing it in RAM.
I'm also using pandas for statistical function.
I need to have access on the entire dataset because many statistical functions needs the entire dataset to return credible results.
I'm using PyDev (with interpreter Python 3.4) on LiClipse with Windows 10.