[Numpy-discussion] Change in scalar upcasting rules for 1.6.x?
Charles R Harris
Mon Feb 13 23:27:13 CST 2012
On Mon, Feb 13, 2012 at 10:01 PM, Travis Oliphant <email@example.com>wrote:
> On Feb 13, 2012, at 10:14 PM, Charles R Harris wrote:
> On Mon, Feb 13, 2012 at 9:04 PM, Travis Oliphant <firstname.lastname@example.org>wrote:
>> I disagree with your assessment of the subscript operator, but I'm sure
>> we will have plenty of time to discuss that. I don't think it's correct to
>> compare the corner cases of the fancy indexing and regular indexing to the
>> corner cases of type coercion system. If you recall, I was quite nervous
>> about all the changes you made to the coercion rules because I didn't
>> believe you fully understood what had been done before and I knew there was
>> not complete test coverage.
>> It is true that both systems have emerged from a long history and could
>> definitely use fresh perspectives which we all appreciate you and others
>> bringing. It is also true that few are aware of the details of how things
>> are actually implemented and that there are corner cases that are basically
>> defined by the algorithm used (this is more true of the type-coercion
>> system than fancy-indexing, however).
>> I think it would have been wise to write those extensive tests prior to
>> writing new code. I'm curious if what you were expecting for the output
>> was derived from what earlier versions of NumPy produced. NumPy has
>> never been in a state where you could just re-factor at will and assume
>> that tests will catch all intended use cases. Numeric before it was not
>> in that state either. This is a good goal, and we always welcome new
>> tests. It just takes a lot of time and a lot of tedious work that the
>> volunteer labor to this point have not had the time to do.
>> Very few of us have ever been paid to work on NumPy directly and have
>> often been trying to fit in improvements to the code base between other
>> jobs we are supposed to be doing. Of course, you and I are hoping to
>> change that this year and look forward to the code quality improving
>> Thanks for all you are doing. I also agree that Rolf and Charles
>> have-been and are invaluable in the maintenance and progress of NumPy and
>> SciPy. They deserve as much praise and kudos as anyone can give them.
> Well, the typecasting wasn't perfect and, as Mark points out, it wasn't
> commutative. The addition of float16 also complicated the picture, and user
> types is going to do more in that direction. And I don't see how a new
> developer should be responsible for tests enforcing old traditions, the
> original developers should be responsible for those. But history is
> history, it didn't happen that way, and here we are.
> That said, I think we need to show a little flexibility in the corner
> cases. And going forward I think that typecasting is going to need a
> No argument on any of this. It's just that this needs to happen at NumPy
> 2.0, not in the NumPy 1.X series. I think requiring a re-compile is
> far-less onerous than changing the type-coercion subtly in a 1.5 to 1.6
> release. That's my major point, and I'm surprised others are more
> cavalier about this.
> New developers are awesome, and the life-blood of a project. But, you
> have to respect the history of a code-base and if you are re-factoring code
> that might create a change in corner-cases, then you are absolutely
> responsible for writing the tests if they aren't there already. That is
> a pretty simple rule.
> If you are changing semantics and are not doing a new major version number
> that you can document the changes in, then any re-factor needs to have
> tests written *before* the re-factor to ensure behavior does not change.
> That might be annoying, for sure, and might make you curse the original
> author for not writing the tests you wish were already written --- but it
> doesn't change the fact that a released code has many, many tests already
> written for it in the way of applications and users. All of these are
> outside of the actual code-base, and may rely on behavior that you can't
> just change even if you think it needs to change. Bug-fixes are different,
> of course, but it can sometimes be difficult to discern what is a "bug" and
> what is just behavior that seems inappropriate.
> Type-coercion, in particular, can be a difficult nut to crack because
> NumPy doesn't always control what happens and is trying to work-within
> Python's stunted type-system. I've often thought that it might be easier
> if NumPy were more tightly integrated into Python. For example, it would
> be great if NumPy's Int-scalar was the same thing as Python's int. Same
> for float and complex. It would also be nice if you could specify scalar
> literals with different precisions in Python directly. I've often wished
> that NumPy developers had more access to all the great language people who
> have spent their time on IronPython, Jython, and PyPy instead.
Going on about casting, somewhere I might still have a table I generated
back around 1.3. Also, Numpy has a mixed C/Numarray system that leads to
some problems, so that should be rationalized at some point.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion