[SciPy-user] Save a list of arrays

Gabriel Beckers beckers@orn.mpg...
Wed Apr 22 08:53:06 CDT 2009


If you want your data saved in a generic way, you could use PyTables
( http://www.pytables.org/ ) to save your arrays in hdf5 format.

A script like the one below already does the trick, although it could be
greatly improved in terms of compression (look into pytables' CArray).
Also if the number of arrays gets really large you have to adapt the
approach, and avoid saving everything in root.

It would be trivial to write a function to read. You can also look at
the contents of the file visually by using ViTables
( http://vitables.berlios.de/ )

Gabriel

------------------------------------------
import numpy
import tables

def write_arraylist(arraylist, filename):
    f = tables.openFile(filename, 'w')
    for i, array in enumerate(arraylist):
        f.createArray(f.root, 'a%d'%i, array)
    f.close()
    
array1 = numpy.ones(2);
array2 = numpy.ones(5);
array3 = numpy.ones(1000);
arraylist = [array1, array2, array3]

write_arraylist(arraylist, 'testfile.h5')





On Wed, 2009-04-22 at 15:09 +0200, Mathieu Dubois wrote:
> [I have posted this message this morning but apparently it is stuck 
> somewhere - sorry for multi-posting]
> 
> Hello,
> 
> I would like to save a list of arrays (each one has a different shape) 
> to a file.
> For instance:
>  >>> array1 = numpy.ones(2);
>  >>> array2 = numpy.ones(5);
>  >>> array3 = numpy.ones(1000);
>  >>> list = [array1, array2, array3]
> 
> As my arrays are huge (each one contains several thousands values) I 
> would like a compressed file.
> numpy.savez would be perfect (because it produces an archive of binary 
> files) but unfortunately numpy.savez(list) doesn't work because savez 
> needs each array individually.
> 
> So what's the best way to do that?
> 
> Maybe I could create an archive file in Python (using ZipFile) save each 
> array in a npy file and then write each file in the archive. But then, 
> will numpy.load be able to open the file?
> 
> Any help appreciated.
> Kind regards,
> Mathieu
> _______________________________________________
> SciPy-user mailing list
> SciPy-user@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
-- 
Dr. Gabriël J.L. Beckers
        
Max Planck Institute for Ornithology
Department of Behavioural Neurobiology

Web: http://www.gbeckers.nl




More information about the SciPy-user mailing list