[Numpy-discussion] Controlling the way a module is imported

Robert Kern robert.kern@gmail....
Tue May 27 01:14:13 CDT 2008

On Mon, May 26, 2008 at 11:01 PM, David Cournapeau
<david@ar.media.kyoto-u.ac.jp> wrote:
> Lisandro Dalcin wrote:
>> David, I've implemented something like this for petsc4py.
>> Basically, I want to be able to build extension modules accessing
>> PETSc for different configuration options (for example, optimized
>> versus debug), when using PETSc, it is normal to select the build
>> variant by a $PETSC_ARCH environ var.
> Yes, that's another solution I forgot to mention.
>> If you think this fits your needs, then I'll help you. Buth then I'll
>> need to know a bit more about the directory tree structure of your
>> package and your extension modules.
> Thanks, but I already have the code to dynamically load a module from a
> string. My problem is how to control it, and how we want to do this kind
> of things. Up to know, scipy have never depended on env variables or
> configuration files, so I don't want to introduce this kind of things
> without discussing it first.

There are actually a couple of places, like scipy.misc.imshow(). I
tend to dislike libraries automatically reading configuration
(including environment variables). However, imshow() is not really a
library function so much as a function for the interactive
interpreter, so I don't mind so much.

I would definitely *not* use environment variables as the sole control
of FFT backend selection. If I have to restart my IPython session just
because I forgot to set the right environment variable, I will be very

For FFTs, I would probably keep the backend functions in a
module-level list. On import, the package would try to import the
backends it knows could be bundled in scipy.fftpack and insert them
into the list. Probably, these would just be the optimized versions;
the default FFTPACK versions would be kept separate. If an import
fails, it would be a good idea to format the traceback into a string
and store it in an accessible location for debugging unintentional
ImportErrors. Each API function (e.g. rfft()) would check its list for
an optimized implementation and use it or else fall back to the
default. Each implementation module (e.g. scipy.fftpack._fftw3, or
whatever you have named them; I haven't reviewed your code, yet) would
have an explicit registration function that puts its implementations
(all or a subset, e.g. _fftw3.register() or _fftw3.register('fft',
'rfft')) at the top of the lists. One needs to think about how to
handle power-of-two only implementations. This is probably the only
real special case we have to handle, so we can probably do it
explicitly instead of coming up with an overly generic solution. I
wouldn't bother trying to persist this information. If anyone wants to
persist their preferences, then can write a utility function that does
exactly what they need.

That's the five-minute overview of what I would try.

Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco

More information about the Numpy-discussion mailing list