[Numpy-discussion] a possible way to implement a plogin system

[David Cournapeau]

Lisandro Dalcin wrote:
>
> Perhaps cleaner is not the right word. I actually believe that is far
> more portable, regarding the oddities of dlopening in different
> platforms.

I am sorry, I don't understand why a struct is less sensitive to 
platform issues than function pointers when dynamically loading them. 
Could you give more details ?

>
> Well, that is the same mechanism NumPy uses for exporting its C API to
> extensions modules. Look at generated header  '__multiarray_api.h' in
> function '_import_array()' in your numpy installation. This is the
> standard, portable, and (as Python doc says) recommended way to expose
> C API's from extensions modules.

Yes, but this is not for exposing C python extensions ! This is for pure 
C plugins, and as such, we have different requirements; in particular, 
we don't need to hide api (the struct thing is partly useful because iso 
C does not have a namespace concept outside the compilation unit with 
static). Again, the use cases I have in mind are multiple 
implementations of the same core functions,  selected at runtime (sse, 
sse2, mmx, etc...). Adding all the python mechanism just to load symbols 
does not sound like simplification to me: using the python C API is 
heavy, you have to start thinking about all kind of issues (ref 
counting, etc...).

>
> Please note that all my concerns about recommending you not to use
> dlopen, is just to save you from future headaches!!!

I implemented and tested the API for mac os x, windows and dlopen which 
covers 99 % of our users I think. Note that we can copy the actual code 
in python sources if we want to.

> .  See yourself at
> row 17 of table here
> <http://www.fortran-2000.com/ArnaudRecipes/sharedlib.html> . And that
> table does not cover stuff like RTLD_GLOBAL flags to dlopen (or
> equivalent).

I do not pretend to know about all the platforms idiosyncrasies, but I 
am aware of the different aspects. In particular, note that most of the 
table is related to building, which I already took care of when starting 
numscons :) Also, I don't think we will use the system outside a few 
majors platforms (typically,  I don'tsee anyone implementing ufunc core 
loops in alpha assembly for True64 :) ).

cheers,

David