[Numpy-discussion] numpy.savez does /not/ compress!?
Tue Jun 8 04:40:59 CDT 2010
>2010/6/8 Hans Meine <email@example.com>:
> I just wondered why numpy.load("foo.npz") was so much faster than loading
> (gzip-compressed) hdf5 file contents, and found that numpy.savez did not
> compress my files at all.
> But is that intended? The numpy.savez docstring says "Save several arrays
> into a single, *compressed* file in ``.npz`` format." (emphasis mine), so I
> guess this might be a bug, or at least a missing feature. In fact, the
> implementation simply uses the zipfile.ZipFile class, without specifying the
> 'compression' argument to the constructor.
> >From http://docs.python.org/library/zipfile.html :
>> `compression` is the ZIP compression method to use when writing the archive,
>> and should be ZIP_STORED or ZIP_DEFLATED; unrecognized values will cause
>> RuntimeError to be raised. If ZIP_DEFLATED is specified but the zlib module
>> is not available, RuntimeError is also raised. The default is ZIP_STORED.
The savez docstring should probably be clarified to provide this information.
I guess that the default (uncompressed Zip) is used because specifying
the compression as ZIP_DEFLATED requires zlib to be installed on the
system (see zipfile.ZipFile docstring).
More information about the NumPy-Discussion