[SciPy-user] Fast saving/loading of huge matrices
Thu Apr 19 15:38:57 CDT 2007
On Thu, Apr 19, 2007 at 02:19:46PM -0500, Ryan Krauss wrote:
> The cPickle approach seems fast, I just wish there was some way to
> make the files smaller. Is there a good way to do this that doesn't
> slow down the read time too much?
The problem with pickle is that the compatibility is not garantied from
one version of modules to another. I would use pytables for everything
else than temporary storage.
Another good thing about pytables is that the file is standard and can be
read by many other programs. I made the decision for our lab to
standardise on hdf5, even without knowing that it was one of the fastest
IO useable with scipy.
Thanks Cárabos for pytables !
More information about the SciPy-user