[Numpy-discussion] Nasty bug using pre-initialized arrays

Zachary Pincus zpincus@stanford....
Sat Jan 5 11:56:52 CST 2008


> There are two related hierarchies of datatypes: different kinds  
> (integer,
> floating point, complex floating point) and different precisions  
> within a given
> kind (int8, int16, int32, int64). The term "downcasting" should  
> probably be
> reserved for the latter only.
>
> It seems to me that Zach and Scott are defending downcasting of  
> precisions
> within a given kind. It does not necessarily follow that the  
> behavior we choose
> for dtypes within a given kind should be the behavior when we are  
> dealing with
> dtypes across different kinds. We can keep the precision  
> downcasting behavior
> that you want while raising an error when one attempts to assign a  
> complex
> number into a floating point array.

Actually, my points were explicitly about cross-kind conversions! A  
lot of image-processing code features heavy use of converting integer  
images to float for intermediate calculations, and then rescaling and  
down-converting back to the original type for display, etc.

In fact, scipy.ndimage makes use of this, allowing for operations  
with output into user-specified arrays, or arrays of user-specified  
dtype, while (I believe) carrying out some of the intermediate  
calculations at higher precision.

As such, pretty much any code that takes a user-specified array and  
assigns to it the result of a (potentially) mixed-mode calculation  
would need to be changed from:

A[i] = calculate(B)
to
A[i] = calculate(B).astype(A.dtype)

with the proliferation of temp arrays that that entails.

I think, but am not sure, that there is a lot of code out there that  
does this, intentionally, which would be broken by this change.  
Explicit is indeed better than implicit, but in this case, finding  
all of the places where mixed-mode conversions happen and tracking  
them down could be a pain on the same scale as const chasing in C++,  
where fixing the error in one place makes it reappear elsewhere in  
the chain of usage, leading to long, painful, and often totally  
pointless debugging sessions.

In the specific case of converting from complex to anything else, I  
can see the desire for a warning to prevent data loss. But converting  
from float to integer is a lot more routine and is used a lot in the  
kind of work that I do, at least.

Zach


More information about the Numpy-discussion mailing list