[Numpy-discussion] Getting C-function pointers from Python to C
Wed Apr 11 16:23:22 CDT 2012
>>> I really hope we can find some project-neutral common ground, so that lots of tools (Cython, f2py, numba, C extensions in NumPy and SciPy) can agree on how to "unbox callables".
>>> A new extension type in NumPy would not fit this bill I feel. I've created a specification for this; if a number of projects (the ones mentioned above) agree on this or something similar and implement support, we could propose a PEP and do it properly once it has proven itself.
>>> In Cython, this may take the form
>>> def call_callback(object func):
>>> cdef double (*typed_func)(int)
>>> typed_func = func
>>> return typed_func(4)
>>> ...it would be awesome if passing a Numba-compiled function just worked in this example.
>> Yes, I think we should go the Python PEP route. However, it will take some time to see that to completion (especially with ctypes already in existence). Dag, this would be a very good thing for you to champion however ;-)
> I was NOT proposing a PEP.
> The spec is created so that it can be implemented *now*, by the tools
> "we" control (and still be very efficient). A "sci-PEP", if you will; a
> mutual understanding between Cython, NumPy, numba (and ideally f2py,
> which already has something similar, if anyone bothers).
> When this is implemented in all the tools we care about, we can propose
> something even nicer as a PEP, but that's far down the road; it'll be
> another couple of years before I'm on Python 3.
Perfect :-) We are on the same page....
>> In the mean-time, I think we could do as Robert essentially suggested and just use Capsule Objects around an agreed-upon simple C-structure:
>> int id /* Some number that can be used as a "type-check" */
>> void *func;
>> char *string;
>> We can then just create some nice functions to go to and from this form in NumPy ctypeslib and then use this while the Python PEP gets written and adopted.
> What is not clear to me is how one get from the Python callable to the
This varies substantially based on the tool. Numba would do it's work and create the capsule object using it's approach. Cython would use a different approach.
I would also propose to have in NumPy some basic functions that go back-and forth between this representation, ctypes, and any other useful representations that might emerge.
> Or do you simply intend to pass a non-callable capsule as an argument in
> place of the callback?
I had simply intended to allow a non-callable capsule argument to be passed in instead of another call-back to any SciPy or NumPy function that can take a raw C-function pointer.
More information about the NumPy-Discussion