[Numpy-discussion] ragged array implimentation
Thu Mar 10 13:29:00 CST 2011
On 3/10/11 9:51 AM, Francesc Alted wrote:
> A Thursday 10 March 2011 18:05:11 Christopher Barker escrigué:
>> NOTE: this looks like it could use a "growable" numpy array, much
>> like one I've written before -- maybe it's time to revive that
>> project and use it here, fixing some performance issues while I'm at
> A growable array would be indeed very useful in many situations. My
> carray project  is another attempt to create such an object. It
> grows (or shrink) arrays by adding (removing) chunks, so they are fast
> beasts for performing these operations. I suppose this is more or less
> like your approach.
I don't think so -- my approach is a lot simpler that I think carray is:
it acts much like a python list:
1) it pre-allocates extra space when an array is created.
2) when you append, it uses that extra space
3) when the extra space is used up, it re-allocates the entire array,
with some more extra room
And to keep it really simple, I'm using a numpy array internally, and
np.ndarray.resize to grow it. It can only be grown on the first axis
IIUC, carray is doing a lot more smarts about keeping the array in
chunks that can be individually manipulated, which is very cool.NOt to
mention the compression!
> I also thought about implementing ragged arrays in carray, like those
> you are suggesting, but not sure when they will be ready for public
That would be very cool, and I think you are right, the facilities in
carray could make a really nice ragged array.
> Perhaps it is a good time to join efforts and create a nice 'growable'
> array package?
Sure, but I don't know that I have much to offer, except maybe some test
and timing code.
By the way, it would be great to have a growable array that could be
efficiently used in Cython code -- so you could accumulate a bunch of
native C datatype data easily.
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
More information about the NumPy-Discussion