[Numpy-discussion] Re: Numeric life as I see it
oliphant at ee.byu.edu
Wed Feb 9 20:12:05 CST 2005
> Martin v. Löwis wrote:
> The PEP should list the options, include criteria
>> for selection, and then propose a choice. People can then discuss
>> whether the list of options is complete (if not, you need to extend
>> it), whether the criteria are agreed (they might be not, and there
>> might be difficult consensus, which the PEP should point out), and
>> whether the choice is the right one given the criteria (there should
>> be no debate about this - everybody should agree factually that the
>> choice meets the criteria best).
> Unrealistic. I think it is undisputed that there are people with
> irreconcilably different needs. Frankly, we spent many, many months on
> the design of Numeric and it represents a set of compromises already.
> However, the one thing it wouldn't compromise on was speed, even at
> the expense of safety. A community exists that cannot live with this
> compromise. We were told that the Python core could also not live with
> that compromise.
I'm not sure I agree. The ufuncobject is the only place where this
concern existed (should we trip OverFlow, ZeroDivision, etc. errors
durring array math). Numarray introduced and implemented the concept
of error modes that can be pushed and popped. I believe this is the
right solution for the ufuncobject.
One question we are pursuing is could the arrayobject get into the core
without a particular ufunc object. Most see this as sub-optimal, but
maybe it is the only way.
> Over the years there was pressure to add safety, convenience,
> flexibility, etc., all sometimes incompatible with speed. Numarray
> represents in some sense the set of compromises in that direction,
> besides its technical innovations. Numeric / Numeric3 represents the
> need for speed camp.
I don't see numarray as representing this at all. To me, numarray
represents the desire to have more flexible array types (specifically
record arrays and maybe character arrays). I personally don't see
Numeric3 as in any kind of "need for speed" camp either. I've never
liked this distinction, because I don't think it represents a true
dichotomy. To me, the differences between Numeric3 and numarray are
currently more "architectural" than implementational.
Perhaps you are referring to the fact that because numarray has several
portions written in Python it is "more flexible" or "more convenient"
but slower for small arrays. If you are saying that then I guess
Numeric3 is a "need for speed" implementation, and I apologize for not
> I think it is reasonable to suppose that the need for speed piece can
> be wrapped suitably by the need for safety-flexibility-convenience
> facilities. I believe that hope underlies Travis' plan.
If the "safety-flexibility-convenience" facilities can be specified,
then I'm all for one implementation. Numeric3 design goals do not go
against any of these ideas intentionally.
> The Nummies (the official set of developers) thought that the Numeric
> code base was an unsuitable basis for further development. There was
> no dissent about that at least. My idea was to get something like what
> Travis is now doing done to replace it. I felt it important to get
> myself out of the picture after five years as the lead developer
> especially since my day job had ceased to involve using Numeric.
Some of the parts needed to be re-written, but I didn't think that meant
moving away from the goal to have a single C-type that is the
arrayobject. During this process Python 2.2 came out and allowed
sub-classing from C-types. As Perry mentioned, and I think needs to be
emphasized again, this changed things as any benefit from having a
Python-class for the final basic array type disappeared --- beyond ease
of prototyping and testing.
> However, removing my cork from the bottle released the unresolved
> pressure between these two camps. My plan for transition failed. I
> thought I had consensus on the goal and in fact it wasn't really
> there. Everyone is perfectly good-willed and clever and trying hard to
> "all just get along", but the goal was lost. Eric Raymond should
> write a book about it called "Bumbled Bazaar".
This is an accurate description. Fortunately, I don't think any
ill-will exists (assuming I haven't created any with my recent
activities). I do want to "get-along." I just don't want to be silent
when there are issues that I think I understand.
> I hope everyone will still try to achieve that goal. Interoperability
> of all the Numeric-related software (including supporting a 'default'
> plotting package) is required.
Utopia is always out of reach :-)
> Aside: While I am at it, let me reiterate what I have said to the
> other developers privately: there is NO value to inheriting from the
> array class. Don't try to achieve that capability if it costs
> anything, even just effort, because it buys you nothing. Those of you
> who keep remarking on this as if it would simply haven't thought it
> through IMHO. It sounds so intellectually appealing that David Ascher
> and I had a version of Numeric that almost did it before we realized
> our folly.
I appreciate some of what Paul is saying here, but I'm not fully
convinced that this is still true with Python 2.2 and up new-style
c-types. The concerns seem to be over the fact that you have to
re-implement everything in the sub-class because the base-class will
always return one of its objects instead of a sub-class object.
It seems to me, however, that if the C methods use the object type
alloc function when creating new objects then some of this problem is
avoided (i.e. if the method is called with a sub-class type passed in,
then a sub-class type gets set).
Have you looked at how Python now allows sub-classing in C? I'm not an
expert here, but it seems like a lot of the problems you were discussing
have been ameliorated. There are probably still issues, but....
I will know more when I seen what happens with a Matrix Object
inheriting from a Python C-array object.
I'm wondering if anyone else with more knowledge about new-style c-types
here and show me the error of my thinking.
More information about the Numpy-discussion