[Numpy-discussion] Porting strategy for py3k

Dag Sverre Seljebotn dagss@student.matnat.uio...
Sun May 3 11:24:47 CDT 2009

David Cournapeau wrote:
> On Fri, Apr 24, 2009 at 4:49 PM, Dag Sverre Seljebotn
>> One thing somebody *could* work on rather independently for some hours
>> is proper PEP 3118 support, as that is available in Python 2.6+ as well
>> and could be conditionally used on those systems.
> Yes, this could be done independently. I am not familiar with PEP
> 3118; from the python-dev ML, it looks like the current buffer API has
> some serious shortcomings, I don't whether this implies to numpy or
> not. Do you have more on this ?

Not sure what you refer to ... I'll just write more and hope it answers
your question.

The difference with PEP 3118 is that many more memory models are supported
(beyond 1D contiguous). All of NumPy's strided arrays are very easy to
expose (after all, Travis Oliphant did the PEP), with no information lost
(i.e. the dtype, shape and strides can be communicated). This means that
one should in Python 3/2.6+ be able to use other CPython libraries (like
image libraries etc.) in a seamless fashion with NumPy arrays without
those libraries having to know about NumPy as such; they can simply
request a strided view of the data through PEP 3118.

To support clients one mainly has to copy out information that is already
there into a struct when requested. The one small challenge is creating a
format string for the buffer dtype (which is incompatible with the current
string representations of dtype).

In addition it would be natural to act as a client, so that calling
"np.array(obj)" (and/or np.view?) would acquire the data through PEP 3118.
There is a class of buffers which doesn't fit in NumPy's memory model
(e.g. pointers to rows of a matrix) and for which a copy would have to be
made, but a lot of them could be used through a direct view as well.

Dag Sverre

More information about the Numpy-discussion mailing list