[Numpy-discussion] Exported symbols and code reorganization.

[David Cournapeau]

Charles R Harris wrote:
>
>
> On 1/7/07, *Robert Kern* <robert.kern at gmail.com 
> <mailto:robert.kern at gmail.com>> wrote:
>
>     Charles R Harris wrote:
>
>
> <snip>
>
>     No, but as with mtrand, most of those arise from the fact that
>     these functions
>     are implemented in files other than the C file that contains the
>     Python module
>     code. In order for them to be called from the Python module code,
>     they need to
>     be exported, IIRMCC (If I Remember My C Correctly). This appears
>     to be true of
>     essentially every other extension module in my site-packages, so I
>     don't think
>     it's going to be much of a problem.
>
>
> I don't think unnecessary public symbols are much of a problem, 
> either, but it bears on the question of breaking up the core numpy 
> files. The parts could be treated in the same way, i.e., compiled 
> separately and linked, or they could be treated as you suggested 
> previously, declared static and the code included into one base file. 
> I suspect the inclusion route is a bit easier from a portability 
> standpoint, long link lists can be a problem and the making of 
> libraries is platform dependent.
>
Bit late at joining the discussion... I started the same thing, 
splitting up those files.

As Travis reminded me in a previous discussion, most symbols in numpy 
extensions are private (eg static functions) because that's the way 
python extension works:

http://docs.python.org/ext/methodTable.html ("The method table must be 
passed to the interpreter in the module's initialization function. The 
initialization function must be named initname(), where name is the name 
of the module, and should be the only non-static item defined in the 
module file:")

By not declaring them static, you are explicitly breaking this. Now, I 
don't see any other reason than avoiding polluting the global namespace 
with the potential to have clashing names (I don't think numpy is likely 
to be big enough so that the cost of loading the shared libraries would 
be prohibitive); but my understanding is that the whole point of having 
a list of pointer of functions when exporting a C Api from a python 
module is precisely for this same reason, eg using static, because 
that's the only portable way to avoid name clashes. The following page 
explains it fairly clearly:

http://docs.python.org/ext/using-cobjects.html

There are other, more elegant solutions to avoid name clashes, but they 
are platform and/or compiler dependent. I exposed the ones I found/heard 
about in a previous email (gcc specific, but I know at least MS compiler 
and Sun C compiler have a similar capability):

"

One elegant solution for this is non portable, unfortunately: recent
gcc version has this functionality called new C++ visibility support,
which also works for C source files.

http://gcc.gnu.org/wiki/Visibility

This file explains the different ways of limiting symbols in dso available

http://people.redhat.com/drepper/dsohowto.pdf

Having several include of C files is the easiest way, and I guess this
would be the safest way to start splitting source files. A better way
can always be used after anyway, I guess."

cheers, 

David