[Numpy-discussion] Numeric 24.0

Travis Oliphant oliphant at ee.byu.edu
Tue Apr 5 22:26:38 CDT 2005


Michiel Jan Laurens de Hoon wrote:

> Travis Oliphant wrote:
>
>> I'd like to release a Numeric 24.0  to get the array interface out 
>> there.   There are also some other bug fixes in Numeric 24.0
>
>
>
> About the array interface, my feeling is that while it may be helpful 
> in the short run, it is likely to damage SciPy in the long run. 


Well, I guess we'll just have to see.   Again, I see the array interface 
as important for talking to other modules that may not need or want the 
"full power" of a packed array module like scipy.base is. 

> The array interface allows different array implementations to move in 
> different directions. These different implementations will be 
> compatible with respect to the array interface, but incompatible 
> otherwise (depending on the level of self-restraint of the developers 
> of the different array implementations). So in the end, extension 
> modules will be written for a specific array implementation anyway. At 
> this point, Numerical Python is the most established and has most 
> users. Numarray, as far as I can tell, keeps closer to the Numerical 
> Python tradition, so maybe extension modules can work with either one 
> without further modification (e.g., pygist seems to work with both 
> Numerical Python and numarray). 

> But SciPy has been moving away (e.g. by replacing functions by methods). 


Michiel, you seem to want to create this impression that "SciPy" is 
"moving away."  I'm not sure of your motivations.   But, since this is a 
public forum, I have to restate emphatically, that "SciPy" is not 
"moving away from Numeric."  It is all about bringing together the 
communities.  For the 5 years that scipy has been in development, it has 
always been about establishing a library of common routines that we 
could all share.   It has built on Numeric from the beginning.  Now, 
there is another "library" of routines that is developing around 
numarray.  It is this very real break that I'm trying to help fix.   I 
have no other "desire" to "move away" or "create a break"  or any other 
such notions that you seem to want to spread.   

That is precisely why I have publically discussed practically every step 
of my work.    You seem to be the only vocal one who thinks that 
scipy.base is not just a replacement for Numeric, but something else 
entirely.    So, I repeat:  **scipy.base is just a new version of 
Numeric with a few minor compatibility issues and a lot of added 
functionality and features**

For example,  despite your claims,  I have  not "replaced" functions by 
methods.  The  functions are still all there just like before.   I've 
simply noticed that numarray has a lot of methods and so I've added 
similar methods to the Numeric object to help numarray users make the 
transition back.

Everything else that I've changed, I've done to bring Numeric up-to-date 
with modern Python versions, and to fix old warts that have sat around 
for years.   If there are problems with my changes, speak up.    Tell me 
what to do to make the new Numeric better.  

> As extension module writers are usually busy people, they may not be 
> willing to modify their code so that it works with SciPy, and even 
> less to maintain two versions of their code, one for Numerical 
> Python/numarray and one for SciPy. 


It's comments like this that make me wonder what you are thinking.  It 
seems to me that you are the only one I've talked to that wants to 
maintain the notion of a "split".  Everybody else, I'm in contact with 
is in full support of merging the two communities behind a single 
scientific array object. 

Every extension module that compiles for Numeric should compile for 
scipy.base.   Notice that full scipy already has a huge number of 
extension modules that needs to compile for scipy.base.   So, I have 
every motivation to make that a painless process.   

> Users who could previously choose to install SciPy as an addition to 
> Numerical Python, now find that they have to choose between SciPy and 
> Numerical Python. As Numerical Python has many more extension 
> packages, I expect that SciPy will end up losing users.


Again,  scipy.base should *replace* Numerical Python for all users 
(except the most adamant who don't seem to want to go with the rest of 
the community).  scipy.base is a new version of Numeric.   On the 
C-level I don't know of any incompatibilities,  on the Python level 
there are a very few (most of them rarely-used typecode character issues 
which a simple search and replace will fix).  


I should emphasize this next point, since I don't seem to be coming 
across very clearly to some people.   As head Numeric developer,  I'm 
stating that **Numeric 24 is the last release that will be called 
Numeric**.   New releases of Numeric will be called scipy.base.  


Of course, I realize that people can do whatever they want with the old 
Numeric code base, but then they will be the ones responsible for 
continuing a "split," because the Numerical Python project at 
sourceforge will point people to install scipy.base.


Help me make the transition as painless as possible, that's all I'm 
asking.   People transitioning from Numeric should have no trouble at 
all as I repeatedly point out.  People transitioning from numarray will 
have a *little* harder time which is why the array interface should help 
out during that process.  It is helping people transition back from 
numarray that is 90% of the reason I've made any changes to the 
internals of Numeric.


I've been a happy and quiet Numeric user and developer for years, but I 
respect the problems that Perry, Rick, Paul, and Todd have pointed out 
with their numarray implementation, and I saw a way to support their 
needs inside of Numeric.  That is the whole reason for my efforts.  I 
wish people would stop trying to make it seem to casual readers of this 
forum that I'm trying to create a "whole new" incompatible system.   
Help me fix the obviously unnecessary incompatibilites where they may 
exist, and help me make automatic transistion scripts to help people 
upgrade painlessly to the newer Numeric. 


I very much appreciate all who voice your concerns.   Michiel, you are 
particularly appreciated because you are voice from a solid Numeric 
user.   I just think that such concerns would be more productive in the 
context of accepting the fact that an upgrade from Numeric to scipy.base 
is going to happen, rather than trying to make it look like some new 
"split" is occurring.    I've received a lot of offline support for the 
Numeric/numarray unification effort that scipy.base is.   It would help 
if  more people could provide public support on this forum so that 
others can see that I'm not just some outsider pushing some random 
ideas, but I am simply someone who decided to sacrifice some time for 
what I think is a very important effort.    It would also help if other 
people who have concerns would voice them (I'm very grateful for those 
who have expressed their concerns) so that we can all address them and 
get on the same page for future development. 


Right now, the CVS version of Numeric3 works reasonably.  It compiles 
and uses the old ufunc objects (which have only been extended to support 
the new types).   I could use a lot of help in finding bugs.    You can 
also try out the new array scalars to see how they work (math works on 
them now) and also see what may still be missing in their implementation.

>
> Personally I use Numerical Python, and I plan to continue to use it 
> for years to come, so it doesn't matter much to me. I'm just warning 
> that the array interface may be a Trojan horse for the SciPy project.


As long as you realize that as far as I know the other developers of 
Numerical Python are going to be moving to scipy.base, and so you will 
be using obsolete technology, you are free to do as you wish.   But, I 
really hope we can persuade you to join us.  It is much better if we 
work together. 


-Travis










More information about the Numpy-discussion mailing list