[SciPy-User] Problem with loading large MAT files

Matthew Brett matthew.brett@gmail....
Wed Mar 2 21:22:16 CST 2011


Hi,

On Mon, Feb 21, 2011 at 7:32 AM, Mohammad Abdollahi
<m.abdollahi@gmail.com> wrote:
> Dear List
>
> I have a couple of MAT files around 450 MB of size that apparently they are
> too large for scipy.io.loadmat function. In deed I keep getting this error :
>
>  File
> "/Library/Frameworks/Python.framework/Versions/6.2/lib/python2.6/site-packages/scipy/io/matlab/mio.py",
> line 140, in loadmat
>    matfile_dict = MR.get_variables()
>  File
> "/Library/Frameworks/Python.framework/Versions/6.2/lib/python2.6/site-packages/scipy/io/matlab/mio5.py",
> line 404, in get_variables
>    hdr, next_position = self.read_var_header()
>  File
> "/Library/Frameworks/Python.framework/Versions/6.2/lib/python2.6/site-packages/scipy/io/matlab/mio5.py",
> line 356, in read_var_header
>    stream = StringIO(dcor.decompress(data))
> SystemError: Objects/stringobject.c:4271: bad argument to internal function
>
>
> but everything is fine when I use a file with a size around 360 MB or sth.
> So does anybody know how to fix this ? of course without having to subdivide
> the original MAT file into samller parts.

Further investigation :

http://bugs.python.org/issue8571

- so I believe this is a bug in the Python zlib module.  As far as I
can see it should be fixed by the latest version of python 2.6, and
load works on my machine for python 2.7.  Can you upgrade somehow and
test?

Best,

Matthew


More information about the SciPy-User mailing list