[SciPy-dev] Extensions linking to libraries used by numpy/scipy

Robert Kern robert.kern at gmail.com
Tue Mar 7 00:16:17 CST 2006


Tom Loredo wrote:
> Robert, thanks for the interesting suggestions!
> 
> Robert Kern wrote:
> 
>>Also, I remember asking Pearu to expose the raw function pointer in the fortran
>>object wrapper. The idea was to be able to allow f2py callback functions to be
>>f2py'ed subroutines themselves, and have everything go really, really fast.
>>
>>In [6]: linalg.flapack.dgetrf._cpointer
>>Out[6]: <PyCObject object at 0xd650>
>>
>>I believe this actually gives you the function pointer to DGETRF_. 
> 
> Alas, this is what I see on OS X (10.3.9):
> 
> In [9]: from scipy import linalg
> In [10]: linalg.flapack.dgetrf._cpointer
> AttributeError: 'module' object has no attribute 'flapack'
> 
> In [11]: linalg.lapack.dgetrf._cpointer
> AttributeError: 'module' object has no attribute 'lapack'
> 
> In [12]: linalg.clapack.dgetrf._cpointer
> AttributeError: 'module' object has no attribute 'clapack'                      
> 
> This is presumably related to whatever is behind the scipy.test()
> warnings about missing blas/lapack on OS X that I and others have reported
> here, presumably reflecting the use of Apple veclib stuff.  Any
> ideas on a portable way to try this?

I dunno. I'm using OS X 10.4, although I think I build with ATLAS. I think you
should at least have flapack. Can you successfully call any of the functions
from scipy.linalg? What versions of numpy/scipy are you using? Can you try "from
scipy.linalg import flapack"? What does

In [3]: from numpy import linalg as n_linalg

In [4]: from scipy import linalg as s_linalg

In [5]: n_linalg is s_linalg
Out[5]: False

give you?

>>Or you could
>>just accept the fortran object itself, and access the function pointer from the
>>PyFortranObject structure itself.
>>
>>Ah, now I've rambled myself to the right answer. Don't pay attention to anything
>>but the last sentence of the previous paragraph. AFAICT, this is a new approach,
>>so you can be our guinea pig.
> 
> I don't know how to pursue this.  I'll have to dig more into
> the f2py docs to understand it.

Probably the most straightforward way is to use the ._cpointer, actually. You
can read about the PyCObject structure in the Python documentation, I believe.
My final suggestion probably requires #includeing the fortranobject.h header
from f2py, which makes things a bit more complicated.

> And since I have the attention of the person behind random/mtrand 8-), 
> I am also wrapping some RNGs in Fortran (e.g., multivariate t), and 
> wondering if there is a way to have them call Numpy's "rand" (or 
> standard_normal, etc.).  I thought perhaps I could try your trick, but:
> 
> In [13]: random.rand
> Out[13]: <built-in method rand of mtrand.RandomState object at 0x3e110>
> In [14]: random.rand._cpointer
> AttributeError: 'builtin_function_or_method' object has no attribute '_cpointer'
> 
> I presume this is because mtrand is not built with f2py. 

Indeed it is not. The module itself is written using Pyrex, and I don't think
there is any reasonable hope of getting Pyrex to do a similar trick. One of
these weekends, I am going to rewrite it in pure C and expose an API with a
similar mechanism as numpy's. I think the extension has stabilized enough that
the benefits of keeping it in Pyrex don't outweigh the usefulness of a C API.

Well, maybe that's not quite true. Let me think about it. If anyone wants to
make a stab at this, let me know. I have a few crazy ideas floating in my head now.

-- 
Robert Kern
robert.kern at gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco




More information about the Scipy-dev mailing list