loadmat python memory error
Question: I'm new to Python and I want to import a matlab struct of size 850M to it. I
use "loadmat" but I get a memory error:

    
    
    return self._matrix_reader.array_from_header(header, process)
      File "mio5_utils.pyx", line 624, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header (scipy\io\matlab\mio5_utils.c:5401)
      File "mio5_utils.pyx", line 653, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header (scipy\io\matlab\mio5_utils.c:4849)
      File "mio5_utils.pyx", line 706, in scipy.io.matlab.mio5_utils.VarReader5.read_real_complex (scipy\io\matlab\mio5_utils.c:5578)
      File "mio5_utils.pyx", line 424, in scipy.io.matlab.mio5_utils.VarReader5.read_numeric (scipy\io\matlab\mio5_utils.c:3439)
      File "mio5_utils.pyx", line 360, in scipy.io.matlab.mio5_utils.VarReader5.read_element (scipy\io\matlab\mio5_utils.c:3164)
      File "streams.pyx", line 76, in scipy.io.matlab.streams.GenericStream.read_string (scipy\io\matlab\streams.c:1408)
    MemoryError
    

I'm running python 3.2 on a Windows XP with 3.5G of RAM. Here is my code:

    
    
    from scipy.io import matlab as mio
    mat = mio.loadmat(DIR + '/input.mat')
    

Could you please help me and tell me what I should do to fix this?


Answer: You are probably using 32-bit Python. The maximum limit for all 32-bit
programs (this issue in fact has nothing to do with Python or Scipy) is 2GB
--- how much memory you have installed on the machine does not matter. In
practice, the allocation of large objects starts to fail already earlier, [due
to virtual memory
fragmentation](http://blogs.msdn.com/b/johan/archive/2007/04/19/why-adding-
more-memory-won-t-resolve-outofmemoryexceptions.aspx) (and ~800 MB seems to be
expected as a rule of thumb).

The solution would be to use 64-bit Python instead of the 32-bit one. For
this, you need a 64-bit operating system such as Windows 7 or Linux.

However, if you are stuck with the 32-bit version of Windows XP, there are
[some tricks](http://msdn.microsoft.com/en-
us/library/windows/desktop/bb613473%28v=vs.85%29.aspx) for bumping the 32-bit
memory limit up to 3GB on 32-bit systems, which may help here.

