[Numpy-discussion] Latest Array-Interface PEP

Timothy Hochberg tim.hochberg at ieee.org
Tue Jan 9 11:35:12 CST 2007


On 1/9/07, Christopher Barker <Chris.Barker at noaa.gov> wrote:
>
> Timothy Hochberg wrote:
> > The reason that I ask is that the two projects that I use regularly are
> > wxPython and PIL generally operate on relatively large data chunks and
> > it's not clear that they would see much benefit over this mechanism
> > versus the array protocol.


Let me preface my remarks by saying, that I was initially assuming that this
was meant as a supplement to the earlier array protocol proposal, not a
replacement as Travis subsequently explained.


But is this mechanism any harder?



It doesn't look like it to me. In
> fact, as I have written a tiny bit of Numeric extension code, this looks
>   familiar and pretty easy to work with.


I expect that the old proposal is easier to implement right now. We could
implement the old array protocol in wxPython and have fairly seemless
integration with numpy without any dependencies. To implement the new
protocol, we'd need the C-API. Since that's not in Python at the moment,
we'd have to include implementations of the various functions into wxPython.
I suppose that wouldn't really be that bad assuming they are already
implemented somewhere. It's a bit of a chicken and the egg problem though.


> > I imagine that between us Chris Barker and I could hack together
> > something for wxPython (not that I've asked him aout it).
>
> I'm not sure when I'll find the time, but I do want to do this.
>
> > And code would
> > probably go a long way to convincing people what a great idea this is.
> > However, all else being equal, it'd be a lot easier to do this for the
> > array protocol since there's no extra infrastructure involved.
>
> Is it that much infrastructure?


I'm not sure. It may not be that bad. I'd guess you'd need both an include
file and a source file for the implementations of the functions.

It looks like this would, at the least, require an extra include file.
> If this flies , then that will be delivered with python 2.8?  until then
> (and for older pythons) would various extension writers all need to add
> this extra file to their source? And might we get a mess with different
> versions floating around out there trying to interact?


Possibly. It shouldn't be a big deal if the API is frozen. But I expect the
best thing to get this to work would be to implement this for as many
projects as possible as trial patches before trying to get this into those
projects officially. That way we can get some experience, tweak the API if
necessary, then freeze it and release it officially.

Like I said, I'll help with wxPython. I'm tempted to try with PIL as well,
but I've never looked at the code there, not even tried to compile it, so I
don't know how far I'd get.


> FWIW, the array protocol PEP seems more relevant to what I do since I'm
> > not concerned so much with the overhead since I'm sending big chunks of
> > data back and forth.
>
> That's the biggest issue, but I think a lot of us use a lot of small
> arrays as well -- and while I don't know if it's a performance hit worth
> worrying about, it's always bugged me that is is faster to convert to a
> python list, then pass it in to wxPython than it is to just pass in the
> array directly.


You should have seen it in the old days before I sped it up. Actually I
think you probably did.

Anyway, it seems like wxPython is low hanging fruit in the sense that we
could probably get it done without too much trouble and it would be pretty
easy. It's possible that Robin may not accept the patch until the relevant
code goes into Python, but just having a patch available would be a useful
template for other project and would show the performance gains this
approach would lead to. At least I sure hope so.

Travis: does the code implementing the C API exist already, or is that
something that still needs to be written?

-tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.scipy.org/pipermail/numpy-discussion/attachments/20070109/cdc1c599/attachment.html 


More information about the Numpy-discussion mailing list