[Numpy-discussion] numpy.savez(_compressed) in loop
Mon Oct 29 03:49:20 CDT 2012
Radek Machulka <radek.machulka <at> gmail.com> writes:
> is there a way how to save more arrays into single npy (npz if possible) file
> in loop? Something like:
> fw = open('foo.bar', 'wb')
> while foo:
> arr = np.array(bar)
> np.savez_compressed(fw, arr)
> Or some workaround maybe? I go through hundreds of thousands arrays and can
> not keep them in memory. Yes, I can save each array into single file, but I
> would better have them all in single one.
You can take a look at the implementation of savez_compressed.
The npz files are nothing but zip files with npy format files inside
them, so you should be able to build them up by stuffing files saved
by np.save() into a zipfile.ZipFile.
However, a better option could be to use a another data format such
More information about the NumPy-Discussion