[Numpy-discussion] NPZ format
Sat Oct 2 03:45:45 CDT 2010
2010/10/2 Robert Kern <firstname.lastname@example.org>
> On Fri, Oct 1, 2010 at 02:13, Francesc Alted <email@example.com> wrote:
> > A Thursday 30 September 2010 18:20:16 Robert Kern escrigué:
> >> On Wed, Sep 29, 2010 at 03:17, Francesc Alted <firstname.lastname@example.org>
> > wrote:
> >> > Hi,
> >> >
> >> > I'm going to give a seminar about serialization, and I'd like to
> >> > describe the .npy format. I noticed that there is a variant of it
> >> > called .npz that can pack several arrays in one single file.
> >> >
> >> > However, .npz does not use compression at all and I'm wondering
> >> > what's the reason. I suppose that this is because you don't want
> >> > to loose the possibility to memmap saved arrays, but can someone
> >> > confirm this?
> >> While I suspect it's possible, I'm certain we don't have any code
> >> that actually does it. Most likely the author assumed that it would
> >> be faster (or tested it to be faster with their CPU/hard disk
> >> configuration) to not compress.
> > Thanks, that's good to know. And yes, I'd say that compressing with zip
> > (zlib) would reduce performance for doing I/O, but most probably
> > decompressing from disk media would represent an improvement in terms of
> > time. At any rate, adding compression capability to .npy should be just
> > one parameter away, so perhaps is a good idea adding it.
> Also some design, documentation, format version bump, and (not least)
> code away. ;-)
Oh, indeed :-)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion