[Numpy-discussion] Latest Array-Interface PEP

Christopher Barker Chris.Barker at noaa.gov
Fri Jan 5 15:39:26 CST 2007


Travis Oliphant wrote:
> Christopher Barker wrote:
>> wxPython -- Robin Dunn
>> PIL      -- Fredrik Lundh
>> PyOpenGL -- Who?
>> PyObjC -- would it be useful there? (Ronald Oussoren)
>> MatplotLib (but maybe it's already married to numpy...)
>> PyGtk ?

> It's a good start, but their is also
> 
> PyMedia, PyVoxel, any video-library interface writers, any audo-library 
> interface writers.

right -- I didn't intend that to be a comprehensive list.

> Anybody who wants to wrap/write code that does some kind of manipulation 
> on a chunk of data of a specific data-format.
> 
> There are so many people who would use it that I don't feel qualified to 
> speak for them all.

I think this is key -- we all know that there are a lot of people that 
*could* use it, and we might even say *should* use it. The question that 
I think the core python devs want answered is *will* they use it. That's 
why I suggest that rather than having a bunch of numpy users make 
comments to python-dev, we really need authors of packages like the 
above to make comments to python-dev, saying "I could use this, and I'll 
  *will* use this if it's been put into the standard lib".

I do think there is one issue that does need to be addressed. The 
current buffer protocol already allows modules to share data without 
copying -- but it doesn't provide any description of that data. This 
proposal would provide more description of that data, but still not 
describe it completely -- that's just not possible, so how helpful is 
the additional description -- I think a lot, but others are not convinced.

Some examples, from my use. I use wxPython a fair bit, so I'll use that 
as an example.

Example 1:

One can now currently pass a buffer into a wxImage constructor to create 
an image from existing data. For instance, you can pass in an WxHx3 
numpy array of unsigned bytes in to create a WxH RGB image. At the 
moment, all the wx.Image constructor checks is if you've passed in the 
correct amount of bytes. With this proposal, it could check to see if 
you passed in a WxHx3 array of bytes. How helpful would that be? it's 
still up to the programmer to make sure those bytes actually represent 
what you want. You may catch a few errors, like accidentally passing in 
a 3xWxH array. You also wouldn't have to pass in the size of the Image 
you wanted - which would be kind  of nice. One could also write the 
Image constructor so that it could take a few different shapes of data, 
and do the right thing with each of them.

How compelling are these advantages?

Example 2:

When you need to draw something like a whole lot of pixels or a long 
polyline, you can now pass into wxPython either: a list of (x,y) tuples, 
or any sequence of any (x,y) sequences. A Nx2 numpy array appears as the 
latter, but it ends up being slower than the list of tuples, because 
wxPython has some code to optimize accessing lists of tuples. 
Internally, wxWidgets has drawing methods that accept a Nx2 c-array of 
ints. With the proposed protocol, wxPython could recognize that such an 
array was passed in, and save a LOT of sequence unpacking, type checking 
and converting, etc. It could also take multiple data types -- floats, 
ints, etc, and do the right thing with each of those.

This to me is more compelling than the Image example.

By the way, Robin Dunn has said that he doesn't want to include numpy 
dependency in wxPython, but would probably accept code that did the 
above if it didn't add any dependencies.

Francesc Altet wrote:

> In any case, I think that if the PEP has success, it would suppose an 
> extraordinary leap towards efficient data interchange in applications that 
> doesn't need (or are reluctant to include) NumPy for their normal operation.

That's the question -- is it an extraordinary leap over what you can now 
do with the existing buffer protocol?


Matthew Brett wrote:

> Is there already, or could there be, some sort of consortium of these
> that agree on the features in the PEP?

There isn't now, and that's my question -- what is the best way to 
involve the developers of some of the many packages that we envision 
using this.

- random polling by the numpy devs and users?
- more organized polling by numpy devs (or Travis)
- just a note encouraging them to pipe in on the discussion here?
- a note encouraging them to pipe in on the discussion at python-dev.

I think the PEP has far more chances of success if it's seen as a 
request from a variety of package developers, not just the numpy crowd 
(which, after all, already has numpy)

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


More information about the Numpy-discussion mailing list