[SciPy-dev] numpy/scipy some remarks

[Fernando Perez]

Andrew Jaffe wrote:
> I have some related numpy/scipy questions:
> 
>   - I recently upgraded to the most recent SVN of numpy, without doing
> likewise for scipy. I found that the numpy.test() calls failed in a
> couple of places -- because *scipy* hadn't been updated with the latest
> dtype updates! (I can't reproduce the errors since I've since updated
> scipy.) I thought the whole point of the numpy/scipy split was to avoid
> 'implicit' calls of scipy from numpy, wasn't it?

Now that things are slowing down a little in the frantic internal 
reorganization, I'd really like to have a clear pronouncement on this issue. 
I personally find that the overwriting of the numpy namespace is just not a 
good idea.  To the Python zen's

	explicit is better than implicit

I'd add the corollary

	and predictability beats convenience


I think pkgload() is a step in the right direction, but I remain convinced 
that it should NOT touch the numpy namespace, _at all_.  Consider the 
following scenario:

1. You write some code which uses numpy.foo, benchmark it and write up the 
results for a talk/paper/whatever.

2. Sometime later, in your code you start using a third-party library for some 
ancillary functionality.  Unbeknownst to you, that library calls pkg_load() 
(you had the env var set to -1 so you could use pkg_load interactively, so you 
don't see any warnings).

3. Now, your benchmarks have changed dramatically, but you can't figure out 
why.  Things are probably better, but it's still very disconcerting.  If 
you're really unlucky, instead you now have some bizarre bug (we've seen some 
of those already).  And you start pulling your hair out: tests done in 
isolation with numpy work fine, but your big code (with the _exact same 
calls_) fails.  At this point, python debugging begins to feel like the kind 
of muddy-water-diving that debugging a stray pointer in C is: something is 
going wrong, and yet there seems to be no causal path you can possibly follow.


Anyway, I won't belabor this point any longer.  I'd just like to hear from 
others their opinion on this matter, and if a decision is made to go ahead 
with the overwriting, at least I think the rationale for it should be well 
justified (and be more than "it's convenient").  The fact that over the last 
few weeks we've had several surprised questions on this is, to me, an 
indicator that I'm not the one uncomfortable with this decision.

Best to all,

f