[SciPy-dev] automatic test script for scipy

eric eric at scipy.org
Fri Apr 12 17:40:35 CDT 2002


>
> Hi Eric,
>
> You have done a nice job.

Thanks!  Vinay Sajip did a nice job writing the logging.py tool based on the PEP
also.  It certainly came in handy here.
http://www.red-dove.com/python_logging.html

>
> On Fri, 12 Apr 2002, eric wrote:
>
> > 4.
> > Atlas.  Right now the script is hardcoded to download some atlas libraries I
> > built for RH on PIII.  This is obviously not portable.  Automating the atlas
> > build is also hard because it requires user input.  I think we could replace
> > config.c with our own config.c that just removed user input request and used
the
> > default values.  That wouldn't be hard.  We'd also need to make config.c
emit
> > the directory where it was going to put the library files in an easy to
parse
> > way so that we'd know where to copy them from.
>
> May be the following hack is simpler:
>
> cd /tmp/dir
> rm -rf ATLAS     # must start from a clean src to avoid config problems
> tar xzf /path/to/atlas3.3.14.tar.gz
> cd ATLAS
> python -c 'for i in range(100): print' | make  #everything is default
> ARCH=`python -c 'import glob;print glob.glob("Make*UNKNOWN")[0][5:]'`
> make install arch=$ARCH

That's a very good solution -- except... On some platforms -- like our RH 7.1
machine, there is one question where you have to answer "no" instead of
yes(which is default).  That is because RH uses a 2.96 compiler which doesn't
produce optimal code for ATLAS.  Even when I specify make CC=kgcc and the
config.c file is built with kgcc, the process still detects 2.96 and complains
about it.  We need to figure out a way around this.

However, your fix will work fine on most machines, so I think it is a fine
solution for now.  I'll just have a "binary atlas install" class that we'll use
here.  Your fix for getting the ARCH is good also.

>
> that will build ATLAS libraries to directory lib/$ARCH. Further fixing
> liblapack.a should be simple.

Yes, I have the stuff to download and build the unoptimized lapack stuff in the
code, so th ar -r stuff should be reasonably simple.

>
> Other thing: it is not complete clear to me whether these automatic
> test hooks are only to be used from enthought machines (from a cron
> job) or should they be used by developers as well? Build everything from
> scratch can take hours on others machines.

Please use them whereever you like.  I think now that it is spewing on 20K or so
of output, the mailing list will give us a reasonable feel for how scipy is
doing on multiple platforms.  I'm not sure the mailing list is the best format
for this, but it was quick and dirty and gives everyone access to the data.
Later we can beautify this whole process and perhaps get rid of the mailing
list.

As far as the time, if your not building atlas and have all but the scipy
snapshot in a local file repository, the build process should be less than an
hour -- no?  It is just building ATLAS that will cause machines to grind on for
endless hours.  Also, after the scripts have run once, the second run is much
faster (no compiling).

I'd like to set up some more scenarios such as testing against a machines
current installations also.  This would be faster.  It should also be pretty
simple -- detect the python version, build anything it is missing into some
tempdir (numeric, f2py, atlas, whatever), and then build scipy.  It is all just
logistics.

Also, if others choose to run this, we may want to hack the scripts some so that
machine name, and more diagnostics are returned.

By the way, did you get this to run on your machine at all Pearu?  I'd be
interested to learn what needs to be re-factored to get less specific to our
network.

eric







More information about the Scipy-dev mailing list