[Numpy-discussion] numpy.savez does /not/ compress!?

Scott Sinclair scott.sinclair.za@gmail....
Tue Jun 8 04:40:59 CDT 2010


>2010/6/8 Hans Meine <meine@informatik.uni-hamburg.de>:
> I just wondered why numpy.load("foo.npz") was so much faster than loading
> (gzip-compressed) hdf5 file contents, and found that numpy.savez did not
> compress my files at all.
>
> But is that intended?  The numpy.savez docstring says "Save several arrays
> into a single, *compressed* file in ``.npz`` format." (emphasis mine), so I
> guess this might be a bug, or at least a missing feature.  In fact, the
> implementation simply uses the zipfile.ZipFile class, without specifying the
> 'compression' argument to the constructor.
>
> >From http://docs.python.org/library/zipfile.html :
>> `compression` is the ZIP compression method to use when writing the archive,
>> and should be ZIP_STORED or ZIP_DEFLATED; unrecognized values will cause
>> RuntimeError to be raised. If ZIP_DEFLATED is specified but the zlib module
>> is not available, RuntimeError is also raised. The default is ZIP_STORED.

The savez docstring should probably be clarified to provide this information.

I guess that the default (uncompressed Zip) is used because specifying
the compression as ZIP_DEFLATED requires zlib to be installed on the
system (see zipfile.ZipFile docstring).

Cheers,
Scott


More information about the NumPy-Discussion mailing list