[Scipy-tickets] [SciPy] #1653: scoreatprecentile return wrong values when used on array with NaNs

SciPy Trac scipy-tickets@scipy....
Wed May 2 14:30:06 CDT 2012


#1653: scoreatprecentile return wrong values when used on array with NaNs
-------------------------+--------------------------------------------------
 Reporter:  imrisofer    |       Owner:  somebody   
     Type:  defect       |      Status:  new        
 Priority:  normal       |   Milestone:  Unscheduled
Component:  scipy.stats  |     Version:  0.10.0     
 Keywords:               |  
-------------------------+--------------------------------------------------

Comment(by josefpktd):

 All functions in scipy.stats that don't have nan in the name are not
 designed to work with nans. It doesn't check and the results depend on the
 implementation details, and will be most likely wrong if they don't raise
 an exception.

 The best way to work with nans currently is to convert the array to a
 masked array with numpy.ma.masked_invalid and use the stats.mstats
 functions.

 IIRC, what scoreatpercentile does is to sort the nans to the end of the
 array, and then find the percentiles of the full length array, essentially
 treating nans as a very large number.

 A possible enhancement for the stats functions would be to introduce an
 extra keyword to check what to do with nans or missing values, NaN, when
 they are available from numpy.   {ignore/remove}

-- 
Ticket URL: <http://projects.scipy.org/scipy/ticket/1653#comment:1>
SciPy <http://www.scipy.org>
SciPy is open-source software for mathematics, science, and engineering.


More information about the Scipy-tickets mailing list