[Numpy-discussion] numpy's future (1.1 and beyond): which direction(s) ?

Alexander Michael lxander.m@gmail....
Sat Mar 22 07:46:15 CDT 2008


On Fri, Mar 21, 2008 at 12:35 AM, David Cournapeau
<cournapeau@cslab.kecl.ntt.co.jp> wrote:
>         numpy 1.0.5 is on the way, and I was wondering about numpy's future. I
>  myself have some ideas about what could be done; has there been any
>  discussion behind what is on 1.1 trac's roadmap ?

MaskedArray, although derived from ndarray, doesn't always play nice
with the rest of numpy as evidenced by the need to recreate many of
the numpy "library" functions specifically for MaskedArrays. There are
many surprises, ones_like returns a MaskedArray when given one, but
empty_like and zeros_like do not, and functions like unique include
masked values in the results, etc. Some of these issues might be
considered bugs (and perhaps already fixed), while others result more
from a lack of overall design for "numpy" working with multiple array
types. Maybe I'm missing something because I'm still relatively new to
numpy, so please correct me if I'm wrong. I'm also thinking obliquely
about sparse arrays (and masked spare arrays?). It would be great, in
my opinion, to move towards a design that allows multiple array types
to work more cohesively in the numpy ecosystem. Precisely, a design
that makes it easier to write functions that work on basic ndarrays,
masked arrays (and sparse arrays?) not by special casing each
container, but by expressing their operations using universal
primitives (of course this isn't always possible, but where it is
possible).

What does this mean? Some functions might work better as methods so
that the different array-like containers can special case them (dot,
for instance, could be a candidate). Or perhaps this escapes the
intent of numpy and what I'm really providing is an argument for why
masked or sparse arrays shouldn't be in numpy and this work (to make
functions more container agnostic) should be carried out in scipy, OR
there should be "three" library stacks for each dense, masked dense,
and sparse arrays.

Alex

P.S. Oh yeah, David's ideas about finding accelerator libraries
dynamically sounds great.


More information about the Numpy-discussion mailing list