[Numpy-discussion] calling NumPy from Julia - a plea for fewer macros

Charles R Harris charlesr.harris@gmail....
Sun Feb 17 13:43:20 CST 2013


On Sun, Feb 17, 2013 at 9:12 AM, Steven G. Johnson <stevenj@alum.mit.edu>wrote:

> Dear NumPy developers,
>
> I've been working on a glue package that allows the Julia language
> (http://julialang.org/) to call Python routines easily
>         https://github.com/stevengj/PyCall.jl
> and I'm using NumPy to pass multidimensional arrays between languages.
>
> Julia has the ability to call C functions directly (without writing C
> glue), and I've been exploiting this to write PyCall purely in Julia.
> (This is nice for a number of reasons; besides programming and linking
> convenience, it means that I can dynamically load different Python
> versions on the same machine, and don't need to recompile if e.g. NumPy
> is updated.) However, calling NumPy has been a challenge, because of
> NumPy's heavy reliance on macros in its C API.
>
> I wanted to make a couple of suggestions to keep in mind as you plan for
> NumPy 2.0:
>
> 1) Dynamically linking to NumPy's C API was challenging, to say the
> least.  Assuming you stick with the PyArray_API lookup table of
> pointers, it would be much easier to call from other languages if you
> include e.g. a numpy.core.multiarray._ARRAY_API_NAMES variable in the
> Python module that is a list of strings giving the symbol names
> corresponding to the  numpy.core.multiarray._ARRAY_API pointer.  (Plus
> documentation, of course.)   Currently I need to parse
> __multiarray_api.h to extract this information, which is somewhat hackish.
>
>
It shouldn't be too much work to provide something like that. The current
API is generated, take a look at numpy/core/codegenerators/numpy_api.py.
PR's welcome.


> 2) Please provide non-macro equivalents (exported in the _ARRAY_API
> symbol table or otherwise) of PyArray_NDIM etcetera to access
> PyArrayObject members.  (e.g. call them PyArray_ndim etc.  Note that
> inline functions are not enough, since they are not loadable
> dynamically.)  Right now, the only ways[*] I can see to access this
> information are either to use C glue (which I want to avoid for the
> reasons above) or to call Python to access the __array_interface__
> attribute (which is suboptimal from a performance standpoint).
>
>
There are already functional versions of PyArray_NDIM and some others, put
in as part of a long term project to hide the numpy internals so that we
can modify structures and such at some point. We could use more work in
that direction and would welcome any input/PR's you might offer. The current
functions can be used instead of the macros by putting

#define NPY_NO_DEPRECATED_API NPY_API_VERSION

before any includes. The NPY_API_VERSION serves to mark which functions
were introduced in which numpy version, so as to maintain backward
compatibility with 3'rd party code. See the lines starting at 1377 in
ndarraytypes.h. for currently available functions.

There might also be some useful things in dynd/blaze which, IIRC, support
numpy for some computations. They are located at
https://github.com/ContinuumIO/

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130217/8360c0b9/attachment.html 


More information about the NumPy-Discussion mailing list