[Numpy-discussion] numpy.savez(_compressed) in loop

[Paul Anton Letnes]

On 29. okt. 2012, at 11:29, Radek Machulka wrote:

> Hi,
> 
> is there a way how to save more arrays into single npy (npz if possible) file 
> in loop? Something like:
> 
> fw = open('foo.bar', 'wb')
> while foo:
> 	arr = np.array(bar)
> 	np.savez_compressed(fw, arr)
> fw.close()
> 
> Or some workaround maybe? I go through hundreds of thousands arrays and can 
> not keep them in memory. Yes, I can save each array into single file, but I 
> would better have them all in single one.

As Pauli said, hdf5 is nicer for such things. I foresee:

import h5py

fw = h5py.File('foo.hdf5', 'w')

while foo:
    arr = np.array(bar)
    fw['arr'] = arr

fw.close()

Good luck
Paul