[Numpy-discussion] saving incrementally numpy arrays

David Warde-Farley dwf@cs.toronto....
Mon Aug 10 23:16:50 CDT 2009

On 10-Aug-09, at 11:29 PM, Juan Fiol wrote:

> Hi, I am creating numpy arrays in chunks and I want to save the  
> chunks while my program creates them. I tried to use numpy.save but  
> it failed (because it is not intended to append data). I'd like to  
> know what is, in your opinion, the best way to go. I will put a few  
> thousands every time but building up a file of several Gbytes. I do  
> not want to put into memory all previous data each time

PyTables sounds like a good way to go. If you need to append to arrays  
themselves it can do that too, but it can certainly append new arrays  
to a file.


More information about the NumPy-Discussion mailing list