[SciPy-user] Fast saving/loading of huge matrices

Francesc Altet faltet@carabos....
Fri Apr 20 12:06:08 CDT 2007


El dv 20 de 04 del 2007 a les 16:43 +0000, en/na Pauli Virtanen va
escriure:
> Fri, 20 Apr 2007 08:24:20 +0200, Gael Varoquaux kirjoitti:
> >
> > I agree that pytable lack a really simple interface. Say something that
> > dumps a dic to an hdf5 file, and vice-versa (althought hdf5 -> dic is a
> > bit harder as all the hdf5 types may not convert nicely to python types).
> 
> In a different attempt to make storing stuff in Pytables easier,
> I wrote a library to dump and load any objects directly to HDF5 files
> 
> 	http://www.iki.fi/pav/software/hdf5pickle/index.html
> 
> It uses the pickle protocol to interface with Python, but unrolls
> objects so that they are stored in the "native" Pytables formats, if
> possible, instead of pickled strings.
> 
> It's a bit rought around some edges and a bit slow, but works. (Also, all
> security issues associated with pickling should be remembered...)
> 
> For example:
[snip]

Wow, a very conscientious work. Why you have not advertised it before?
I love to hear about these kind of developments...

Cheers,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth



More information about the SciPy-user mailing list