[SciPy-Dev] adding chkfinite flags to linalg functions

Ralf Gommers ralf.gommers@googlemail....
Sun Sep 18 06:31:21 CDT 2011


On Sat, Sep 17, 2011 at 2:11 PM, Martin Teichmann
<lkb.teichmann@gmail.com>wrote:

> Hi list,
>
> I've been working on qr_multiply, as some of you might have already
> known, and realized that more than 10% of the time is actually spent
> in asarray_chkfinite, at which point I discovered this discussion and
> just wanted to add my two cents.
>
> I think it is a problem that should be tackled, but there are several
> possible solutions:
>
> * just say numpy.asarray_chkfinite=numpy.asarray. That works fine as
> a hack if you know what you're doing
>
> * adding the parameter chkfinite to all functions. That's uncool, I think,
> hard to maintain on the long run.
>

Agreed that it's ugly.

>
> * improving asarray_chkfinite. I think Fabian is working on that.
> actually, the only thing needed would be a function isallfinite, since
> I guess that most of the time is spent in the creation of the logical
> array in isfinite(a).all().
>

Fabian noted at https://github.com/scipy/scipy/pull/48 that he was giving up
for now. He did give some suggestions about possible ways to go, quoting:

- A pure-cython, type-generic function as present in pandas [0] is slow. To
go faster, types have to be specialized. For LAPACK, it might not be a
problem to write 4 similar routines (float, double, complex, double
complex), but it's ugly.

- BLAS i_amax function doesn't propagate NaNs but can be used to detect
{-np.inf, np.inf} extremely fast.

- NumPy has the infrastructure to write functions for several dtypes in a
templated fashion (in C). I haven't tried that.



>
> * adding a ALLFINITE flag to ndarrays. Gaël correctly pointed out that
> this is not as simple as it sounds, but it's nevertheless possible:
> ALLFINITE would have to be a immutable flag at creation time of the
> ndarray, and everytime one does something to the ndarray (possibly
> via a view, to which the ALLFINITE flag has to be copied), we have
> to check everything is finite. This solution would create a lot of work,
> but also nice side-effects: even NaN-aware functions in numpy can
> be very slow on some (actually very common) hardware, accidental
> NaNs can be very annoying.
>

Not impossible, but it would create much bigger problems than it solves.


>
> * adding an "finite" intent into f2py. This is currently my preferred
> solution.
> as f2py often has to copy the data to make it available for FORTRAN,
> checking finiteness could be done while copying with virtually no
> loss of speed.
>

Would this work in all cases? And wouldn't you get a small penalty for cases
where you don't need to use asarray_chkfinite (not sure how often this
happens)?

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-dev/attachments/20110918/3060fbe1/attachment-0001.html 


More information about the SciPy-Dev mailing list