[Numpy-discussion] Exported symbols and code reorganization.
david at ar.media.kyoto-u.ac.jp
Tue Jan 9 02:45:31 CST 2007
Charles R Harris wrote:
> On 1/7/07, *Robert Kern* <robert.kern at gmail.com
> <mailto:robert.kern at gmail.com>> wrote:
> Charles R Harris wrote:
> No, but as with mtrand, most of those arise from the fact that
> these functions
> are implemented in files other than the C file that contains the
> Python module
> code. In order for them to be called from the Python module code,
> they need to
> be exported, IIRMCC (If I Remember My C Correctly). This appears
> to be true of
> essentially every other extension module in my site-packages, so I
> don't think
> it's going to be much of a problem.
> I don't think unnecessary public symbols are much of a problem,
> either, but it bears on the question of breaking up the core numpy
> files. The parts could be treated in the same way, i.e., compiled
> separately and linked, or they could be treated as you suggested
> previously, declared static and the code included into one base file.
> I suspect the inclusion route is a bit easier from a portability
> standpoint, long link lists can be a problem and the making of
> libraries is platform dependent.
Bit late at joining the discussion... I started the same thing,
splitting up those files.
As Travis reminded me in a previous discussion, most symbols in numpy
extensions are private (eg static functions) because that's the way
python extension works:
http://docs.python.org/ext/methodTable.html ("The method table must be
passed to the interpreter in the module's initialization function. The
initialization function must be named initname(), where name is the name
of the module, and should be the only non-static item defined in the
By not declaring them static, you are explicitly breaking this. Now, I
don't see any other reason than avoiding polluting the global namespace
with the potential to have clashing names (I don't think numpy is likely
to be big enough so that the cost of loading the shared libraries would
be prohibitive); but my understanding is that the whole point of having
a list of pointer of functions when exporting a C Api from a python
module is precisely for this same reason, eg using static, because
that's the only portable way to avoid name clashes. The following page
explains it fairly clearly:
There are other, more elegant solutions to avoid name clashes, but they
are platform and/or compiler dependent. I exposed the ones I found/heard
about in a previous email (gcc specific, but I know at least MS compiler
and Sun C compiler have a similar capability):
One elegant solution for this is non portable, unfortunately: recent
gcc version has this functionality called new C++ visibility support,
which also works for C source files.
This file explains the different ways of limiting symbols in dso available
Having several include of C files is the easiest way, and I guess this
would be the safest way to start splitting source files. A better way
can always be used after anyway, I guess."
More information about the Numpy-discussion