[Numpy-discussion] C Extensions, CTypes and "external code & libraries"

[David Cournapeau]

Lou Pecora wrote:
> --- Jon Wright <wright@esrf.fr> wrote:
>
>> Lou Pecora wrote:
>>  >...  This appears to be the way
>>> static and shared libraries work, especially on
>> Mac OS
>>> X, maybe elsewhere.
>> Have you tried linking against a GSL static library?
>> I don't have a mac, 
>> but most linkers only pull in the routines you need.
>> For example, using 
>> windows and mingw:
>>
>> #include <stdio.h>
>> #include <gsl/gsl_sf_bessel.h>
>> int main (void)
>> {  double x = 5.0;
>>     double y = gsl_sf_bessel_J0 (x);
>>     printf ("J0(%g) = %.18e\n", x, y);
>>     return 0; }
>>
>> ...compiles to a.exe which outputs:
>>
>> J0(5) = -1.775967713143382900e-001
>>
>
> Yes, I know about this approach if I am making an
> executable.  But I want to make my code into a shared
> library (my code will not have a main, just the
> functions I write) and, if possible, let my code call
> the GSL code it needs from the C function I write
> (i.e. no python interface).  If what you did can be
> done for a shared library, then that would be great. 
> However, I am ignorant of how to do this.  I will try
> to make my shared library using gcc and then add the
> GSL library using the -l option as someone else
> suggested.  Maybe that will work. 
Oh, I may have misunderstood what you are trying to do then. You just 
want to call a shared library from another shared library ? This is 
possible on any platform supporting shared library (including but not 
limited to mac os x, windows, linux, most not ancient unices).

As Albert said, just do (with gcc):

gcc -shared -o mysharedlib mysharedlib.c -lgsl

This works on mac os X as well as linux (and even windows with mingw). 
If you want to link the gsl statically (so that your own lib does not 
depend on the gsl anymore), you have use a trick to tell gcc to link the 
gsl:

gcc -shared  -o mysharedlib mysharedlib.c -Wl,-Bstatic -lgsl -Wl,-Bdynamic

-Wl is used by gcc to pass option to the linker directly. -Bstatic says 
that all link options after will be static. You have to use -Bdynamic 
after, to avoid linking everything static (like gcc runtime, the C lib, 
which are automatically linked by default by gcc: it is almost always a 
bad idea to statically link those).

In the first case, mysharedlib.so will need libgsl.so:

ldd mysharedlib.so ->         linux-gate.so.1 =>  (0xffffe000)
        libgsl.so.0 => /usr/lib/libgsl.so.0 (0xb7ddb000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7c91000)
        libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7c6b000)
        /lib/ld-linux.so.2 (0x80000000)

In the second case:

        linux-gate.so.1 =>  (0xffffe000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7e60000)
        /lib/ld-linux.so.2 (0x80000000)

I don't know if the second method works on mac os X: since it bypass gcc 
and goes directly to the linker, which is notably different on mac os X, 
you may have to do it differently.
>  I'll report back. 
> I have been searching for info on the right approach
> to this on the Mac, since, as I understand, Mac OS X
> does make a distinction between shared libraries and
> dynamic libraries (which I don't understand fully).  
To be over-simplistic: shared libraries are linked into the executable, 
and all its symbols (function, variables) are solved when you launch the 
executable. A dynamic library is not linked into the executable, and can 
be loaded at anytime during the execution of the executable. Shared 
library are "just" a way to avoid duplicate code, but are totally 
transparent to the code user:

int foo()
{
    return bar();
}

If bar is in a shared lib (libbar.so) or in another object code (bar.o), 
it does not make a difference for you. With a dynamic lib, on most 
unices, you do

hdl = dlopen("libbar.so")
((int)(*bar)()) = dlsym(hdl, "bar");

That is you explicitly load the functions you want to use. Without this 
scheme, you would have to link your extension to the python executable 
when python is built, which is totally impractical of course. IOW, 
dynamic libraries are used for "plugins", things which can be added to 
an executable *after* the executable is built.

On linux (and other unices using the elf binary format), both types of 
libraries are built exactly the same. On mac os X (and windows as well), 
they are not. Again, this is oversimplification, but you don't need to 
know much more in almost all the cases.

cheers,

David