[Numpy-discussion] calling NumPy from Julia - a plea for fewer macros
Steven G. Johnson
Sun Feb 17 10:12:43 CST 2013
Dear NumPy developers,
I've been working on a glue package that allows the Julia language
(http://julialang.org/) to call Python routines easily
and I'm using NumPy to pass multidimensional arrays between languages.
Julia has the ability to call C functions directly (without writing C
glue), and I've been exploiting this to write PyCall purely in Julia.
(This is nice for a number of reasons; besides programming and linking
convenience, it means that I can dynamically load different Python
versions on the same machine, and don't need to recompile if e.g. NumPy
is updated.) However, calling NumPy has been a challenge, because of
NumPy's heavy reliance on macros in its C API.
I wanted to make a couple of suggestions to keep in mind as you plan for
1) Dynamically linking to NumPy's C API was challenging, to say the
least. Assuming you stick with the PyArray_API lookup table of
pointers, it would be much easier to call from other languages if you
include e.g. a numpy.core.multiarray._ARRAY_API_NAMES variable in the
Python module that is a list of strings giving the symbol names
corresponding to the numpy.core.multiarray._ARRAY_API pointer. (Plus
documentation, of course.) Currently I need to parse
__multiarray_api.h to extract this information, which is somewhat hackish.
2) Please provide non-macro equivalents (exported in the _ARRAY_API
symbol table or otherwise) of PyArray_NDIM etcetera to access
PyArrayObject members. (e.g. call them PyArray_ndim etc. Note that
inline functions are not enough, since they are not loadable
dynamically.) Right now, the only ways[*] I can see to access this
information are either to use C glue (which I want to avoid for the
reasons above) or to call Python to access the __array_interface__
attribute (which is suboptimal from a performance standpoint).
Thanks for all your efforts! Any feedback on PyCall would be welcome, too.
[*] A third way would be to parse ndarraytypes.h to extract the format
of the PyArrayObject_fields structure, and use upcoming Julia support
for accessing C struct types to read the fields. This is likely to
require tracking NumPy releases carefully to avoid breakage, however, as
well as involving some care with the PyObject_HEAD macro.
PS. If you want to try out PyCall with NumPy, note that a patch to Julia
is currently required for this to work:
More information about the NumPy-Discussion