[SciPy-dev] issues with scipy and eggs

[Robert Kern]

Andrew Straw wrote:
> Hi,
> 
> Issue 1 - probably a minor bug with scipy distutils
> ===================================================
> 
> Attempting to build scipy (full) using "python setup.py bdist_egg" fails
> on my machine with:
> 
> /usr/bin/g77 -shared
> build/temp.linux-x86_64-2.4/build/src/Lib/fftpack/_fftpackmodule.o
> build/temp.linux-x86_64-2.4/Lib/fftpack/src/zfft.o
> build/temp.linux-x86_64-2.4/Lib/fftpack/src/drfft.o
> build/temp.linux-x86_64-2.4/Lib/fftpack/src/zrfft.o
> build/temp.linux-x86_64-2.4/Lib/fftpack/src/zfftnd.o
> build/temp.linux-x86_64-2.4/build/src/fortranobject.o
> -Lbuild/temp.linux-x86_64-2.4 -ldfftpack -lg2c-pic -o
> build/lib.linux-x86_64-2.4/scipy/fftpack/_fftpack.so
> /usr/bin/ld: cannot find -ldfftpack
> collect2: ld returned 1 exit status
> /usr/bin/ld: cannot find -ldfftpack
> collect2: ld returned 1 exit status
> error: Command "/usr/bin/g77 -shared
> build/temp.linux-x86_64-2.4/build/src/Lib/fftpack/_fftpackmodule.o
> build/temp.linux-x86_64-2.4/Lib/fftpack/src/zfft.o
> build/temp.linux-x86_64-2.4/Lib/fftpack/src/drfft.o
> build/temp.linux-x86_64-2.4/Lib/fftpack/src/zrfft.o
> build/temp.linux-x86_64-2.4/Lib/fftpack/src/zfftnd.o
> build/temp.linux-x86_64-2.4/build/src/fortranobject.o
> -Lbuild/temp.linux-x86_64-2.4 -ldfftpack -lg2c-pic -o
> build/lib.linux-x86_64-2.4/scipy/fftpack/_fftpack.so" failed with exit
> status 1
> 
> However, "python setup.py build bdist_egg" works.  Is there something
> wrong with scipy distutils that is causing this behavior?  I am happy to
> provide more information if requested...

I've found scipy.distutils (and scipy_distutils) to be cranky about computing
the dependencies between build commands. I always use a build script, e.g.:

  python setup.py build_src build_clib build_ext -i --fcompiler=gnu

> Issue 2 - how to use full scipy from an egg?
> ============================================
> 
> I can't figure out the incantation for using scipy (full) from an egg. 
> AFAICT this should just work with "import scipy" because both the
> scipy_core and scipy egg are in my easy-install.pth file.  But doing
> "import scipy" only gets me scipy core.  Furthermore, Robert Kern
> suggested the following, which does apparently find the full scipy egg,
> but it doesn't  load full scipy.

No, you need to add the following to the scipy/__init__.py in both scipy_core
and full scipy:

try:
    __import__('pkg_resources').declare_namespace(__name__)
except ImportError:
    pass

That way, pkg_resources knows to keep looking for the other half.

Also, if both eggs are already on sys.path, then pkg_resources.require() isn't
necessary. Additionally, if they weren't the lone pkg_resources.require('SciPy')
wouldn't find scipy_core.

-- 
Robert Kern
robert.kern at gmail.com

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter