[Numpy-discussion] numpy, Py_ssize_t, cython and 64 bits python 2.4
Dag Sverre Seljebotn
Mon Nov 10 04:32:49 CST 2008
Charles R Harris wrote:
> On Sun, Nov 9, 2008 at 4:29 PM, Dag Sverre Seljebotn
> <firstname.lastname@example.org <mailto:email@example.com>> wrote:
> What the Cython numpy.pxd file does is implementing PEP 3118 ,
> is supported by Cython in all Python versions (ie backported, not
> following any standard). And, in Py_buffer, the strides and shapes are
> Py_ssize_t* (which is also backported as David mentions). So, in order
> to cast the shape and stride arrays and return them in the Py_buffer
> struct, they need to have the datatype defined by the backported PEP
> 3118, i.e. the backported Py_ssize_t, i.e. int.
> So the backported version is pretty much a cython standard?
Yes. The whole thing is described in
http://wiki.cython.org/enhancements/buffer , under the section "Buffer
acquisition and Python versions". Basically the PEP is followed for
Python 2.6+, then some tricks are done to make a similar interface work
for older Python versions.
Note especially that if you NumPy people set Py_TPFLAGS_HAVE_NEWBUFFER
and provide a bf_getbuffer implementation following the exact
(non-backported) , the __getbuffer__ in numpy.pxd will not come into
play on Python 2.6+.
> OTOH, one could also opt for changing how PEP 3118 is backported
> and say
> that "for Python 2.4 we say that Py_buffer has Py_intptr_t* fields
> instead". This would be more work to get exactly right, and would be
> more contrived as well, but is doable if one really wants to get
> rid of
> the extra mallocs.
> This would be the direct way. The check could then be if
> sizeof(npy_intp) != sizeof(Py_intptr_t). That is more reasonable as
> they are supposed to serve the same purpose. If numpy is the only user
> of this interface that is the route I would go. Is there an official
> description of how PEP 3118 is to be backported? I don't know who else
> uses it at the moment.
NumPy is currently the only user that I know of, and is likely to remain
so I think, so making this change is ok. The backport is very
Cython-specific because of the Cython-specific calling convention
(essentially, objects lack the needed slot in earlier Python versions,
and we do not want to pay the price of a dict lookup) and is about as
official as things get with Cython (i.e. a wikipage).
More information about the Numpy-discussion