[Numpy-discussion] ragged array implimentation

Christopher Barker Chris.Barker@noaa....
Mon Mar 7 14:29:35 CST 2011


On 3/7/11 11:18 AM, Francesc Alted wrote:

> Well, I don't think there is such a 'standard' way for dealing with
> ragged arrays, but yes, pytables has support for them.  Creating them is
> easy:
>
> # Create a VLArray:
> fileh = tables.openFile('vlarray1.h5', mode='w')
> vlarray = fileh.createVLArray(fileh.root, 'vlarray1',
>                                tables.Int32Atom(shape=()),
>                                "ragged array of ints",
>                                filters=tables.Filters(1))
> # Append some (variable length) rows:
> vlarray.append(array([5, 6]))
> vlarray.append(array([5, 6, 7]))
> vlarray.append([5, 6, 9, 8])
>
> Then, you can access the rows in a variety of ways, like iterators:
>
> print '-->', vlarray.title
> for x in vlarray:
>      print '%s[%d]-->  %s' % (vlarray.name, vlarray.nrow, x)
>
> -->  ragged array of ints
> vlarray1[0]-->  [5 6]
> vlarray1[1]-->  [5 6 7]
> vlarray1[2]-->  [5 6 9 8]
>
> or via __getitem__, using general fancy indexing:
>
> a_row = vlarray[2]
> a_list = vlarray[::2]
> a_list2 = vlarray[[0,2]]   # get list of coords
> a_list3 = vlarray[[0,-2]]  # negative values accepted
> a_list4 = vlarray[numpy.array([True,...,False])]  # array of bools


> but, instead of returning a numpy array of 'object' elements, plain
> python lists are returned instead.

which gives you the append option -- I can see how that would be 
usefull. Though I'd kind of like to have numpy ufunc/broadcasting 
performance. i.e.:

vlarray * some_constant

Be fast, and work out of the box!

>  More info on VLArray object in:
>
> http://www.pytables.org/docs/manual/ch04.html#VLArrayClassDescr
>
>> is a "vlen array" stored contiguously in netcdf?

great, thanks! that gives me an example of one API I might want to use.

> I don't really know, but one limitation of variable length arrays in
> HDF5 (and hence NetCDF4) is that they cannot be compressed (but that
> should be addressed in the future).

good to know.

Thanks to both you and Jeff, This has given me some things to ponder.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov


More information about the NumPy-Discussion mailing list