[Numpy-discussion] fast numpy i/o
Tue Jun 21 13:01:25 CDT 2011
Neal Becker wrote:
> I'm wondering what are good choices for fast numpy array serialization?
> mmap: fast, but I guess not self-describing?
> hdf5: ?
Should be pretty fast, and self describing -- advantage of being a
standard. Disadvantage is that it requires an hdf5 library, which can b
a pain to install on some systems.
> pickle: self-describing, but maybe not fast?
there is .tofile() and .fromfile() -- should be about as fast as you can
get, not self-describing.
Then there is .save(), savez() and .load() (.npz format)? It should be
pretty fast, and self-describing (but not a standard outside of numpy).
I doubt pickle will ever be your best bet.
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
More information about the NumPy-Discussion