[Numpy-discussion] ufuncs on funny strides; also isnan, isfinite, etc on a variety of dtypes
Thu Apr 1 16:23:11 CDT 2010
On 1 April 2010 14:30, M Trumpis <email@example.com> wrote:
> Hi all,
> disclaimer: pardon my vast ignorance on the subject of ufuncs, that
> will explain the naivety of the following questions
> This morning I was looking at this line of code, which was running
> quite slow for me and making me think
> data_has_nan = numpy.isnan(data_array).any()
> I knew that the array was of type uint8. This raised the question of
> how much slower would a function like isnan() run on integer typed
> data, and furthermore if it's integer typed data then why bother
> looking at the values at all?
Yes, in fact there is no NaN for integer types, so this will never
come out True.
> Actually I realized later that the main slow-down comes from the fact
> that my array was strided in fortran order (ascending strides). But
> from the point of view of a ufunc that is operating independently at
> each value, why would it need to respect striding?
This is a subject that has come up a number of times. In an ideal
world, numpy would be free to run ufuncs in a cache-optimal order. As
it stands, though, numpy does some limited and possibly ill-advised
reordering, and in fact the result of certain operations can depend on
the order in which numpy carries out an operation, so it's kind of a
pain to fix this. But we'd like to.
The conclusion seems to be to do it in two steps: first, declare by
fiat that all operations reading from and writing to the same memory
return the same result as if the result were saved to a new array and
then copied into the result. This changes the result of certain
currently ill-defined operations, and requires real copies in some
cases, but would remove undefined behaviour. The second step, once we
have the freedom to reorder operations, would be to write some ufunc
reordering code that flattens arrays as much as possible and loops
over cache-coherent strides first if possible.
Somebody needs to write the code, though.
> And a last mini question, it doesn't appear that any() is doing short
> circuit evaluation. It runs in appx the same time whether an array is
> sparsely nonzero, fully zero, or fully nonzero.
> Kind regards,
> NumPy-Discussion mailing list
More information about the NumPy-Discussion