[Numpy-discussion] fast numpy i/o
Tue Jun 21 13:22:19 CDT 2011
Neal Becker wrote:
>> I'm wondering what are good choices for fast numpy array serialization?
>> mmap: fast, but I guess not self-describing?
>> hdf5: ?
>> pickle: self-describing, but maybe not fast?
> I think, in addition, that hdf5 is the only one that easily interoperates with
netcdf is another option if you want an open standard supported by
Matlab and the like. I like the netCDF4 package:
Though is can be kind of slow to write (I have no idea why!)
plain old tofile() should be readable by other tools as well, as long as
you have a way to specify what is in the file.
> speaking of hdf5, I see:
> pyhdf5io 0.7 - Python module containing high-level hdf5 load and save
> h5py 2.0.0 - Read and write HDF5 files from Python
There is also pytables, which uses HDF5 under the hood.
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
More information about the NumPy-Discussion