[SciPy-User] Creating smaller .mat files

Matthew Brett matthew.brett@gmail....
Wed Jan 18 16:05:53 CST 2012


Hi,

On Wed, Jan 18, 2012 at 9:48 PM, Aronne Merrelli
<aronne.merrelli@gmail.com> wrote:
>
>
> On Wed, Jan 18, 2012 at 1:54 PM, Jaidev Deshpande
> <deshpande.jaidev@gmail.com> wrote:
>>
>> Hi
>>
>> I use the savemat function in the scipy.io module quite often.
>>
>> Suppose you create a .mat file through Python. Most often this file is
>> unnecessarily large.
>>
>> Even more interesting - when you open that file into a MATLAB workspace
>>
>> >> load myfile.mat
>>
>> And then save it back,
>>
>> >> save myfile.mat
>>
>> the size reduces dramatically! (Even though the data types and
>> precision are kept the same)
>>
>> Why does this happen?
>>
>
> By default, MATLAB compresses the arrays that are written into the file. I
> don't often write MATLAB format files from scipy, but the help for
> scipy.io.matlab.savemat shows this keyword:
>
>     do_compression : bool, optional
>        Whether to compress matrices on write. Default is False.
>
> I would assume that would be equivalent what MATLAB is doing. Perhaps try
> that keyword first? Otherwise, you could look at PyTables, and write HDF5
> files (which also support compression), or have a look at savez_compressed
> in NumPy.

Also, matlab will work out whether your floats are in fact all
integers and store them as such, but the scipy writer does not try and
do that.  It wouldn't be that hard to do, but it could obviously have
an impact on speed.  I guess that would depend on the size of the
arrays.

Best,

Matthew


More information about the SciPy-User mailing list