[Numpy-discussion] saving incrementally numpy arrays

[David Warde-Farley]

On 10-Aug-09, at 11:29 PM, Juan Fiol wrote:

> Hi, I am creating numpy arrays in chunks and I want to save the  
> chunks while my program creates them. I tried to use numpy.save but  
> it failed (because it is not intended to append data). I'd like to  
> know what is, in your opinion, the best way to go. I will put a few  
> thousands every time but building up a file of several Gbytes. I do  
> not want to put into memory all previous data each time

PyTables sounds like a good way to go. If you need to append to arrays  
themselves it can do that too, but it can certainly append new arrays  
to a file.

David