[Scipy-tickets] [SciPy] #1489: fisher_exact throws ValueErrors when row or col is 0, 0

SciPy Trac scipy-tickets@scipy....
Sun Aug 7 08:40:28 CDT 2011


#1489: fisher_exact throws ValueErrors when row or col is 0,0
--------------------------+-------------------------------------------------
 Reporter:  aihardin      |       Owner:  somebody   
     Type:  defect        |      Status:  new        
 Priority:  high          |   Milestone:  Unscheduled
Component:  scipy.stats   |     Version:  0.9.0      
 Keywords:  fisher_exact  |  
--------------------------+-------------------------------------------------

Old description:

> scipy.version.version is '0.9.0rc2'
> numpy.version.version is '1.5.1'
> python is Python 2.7.1 |EPD 7.0-1 (64-bit)| (r271:86832, Dec  3 2010,
> 15:56:20)
> on OS X 10.7
>

> when using stats.fisher_exact I am getting errors that make the function
> unusable for cases where I might have a case where both values in a
> column or a row are 0.
>
> o,p =stats.fisher_exact([[0, 1],[0, 2]]) gives
> ValueError                                Traceback (most recent call
> last)
>
> /Users/aaron/PythonCourse/S9.2_mapping_snps/<ipython console> in
> <module>()
>
> /Library/Frameworks/EPD64.framework/Versions/7.0/lib/python2.7/site-
> packages/scipy/stats/stats.py in fisher_exact(table)
>    2257
>    2258     epsilon = 1 - 1e-4
> -> 2259     if float(np.abs(pexact - pmode)) / np.abs(np.max(pexact,
> pmode)) <= 1 - epsilon:
>    2260         return odssratio, 1
>    2261
>
> /Library/Frameworks/EPD64.framework/Versions/7.0/lib/python2.7/site-
> packages/numpy/core/fromnumeric.py in amax(a, axis, out)
>    1798     except AttributeError:
>    1799         return _wrapit(a, 'max', axis, out)
> -> 1800     return amax(axis, out)
>    1801
>    1802
>
> ValueError: cannot convert float NaN to integer
>
> and I get a different ValueError in other cases:
>
> o,p =stats.fisher_exact([[10, 0],[10, 0]])
> ---------------------------------------------------------------------------
> ValueError                                Traceback (most recent call
> last)
>
> /Users/aaron/PythonCourse/S9.2_mapping_snps/<ipython console> in
> <module>()
>
> /Library/Frameworks/EPD64.framework/Versions/7.0/lib/python2.7/site-
> packages/scipy/stats/stats.py in fisher_exact(table)
>    2257
>    2258     epsilon = 1 - 1e-4
> -> 2259     if float(np.abs(pexact - pmode)) / np.abs(np.max(pexact,
> pmode)) <= 1 - epsilon:
>    2260         return odssratio, 1
>    2261
>
> /Library/Frameworks/EPD64.framework/Versions/7.0/lib/python2.7/site-
> packages/numpy/core/fromnumeric.py in amax(a, axis, out)
>    1798     except AttributeError:
>    1799         return _wrapit(a, 'max', axis, out)
> -> 1800     return amax(axis, out)
>    1801
>    1802
>
> ValueError: axis(=1) out of bounds

New description:

 scipy.version.version is '0.9.0rc2'
 numpy.version.version is '1.5.1'
 python is Python 2.7.1 |EPD 7.0-1 (64-bit)| (r271:86832, Dec  3 2010,
 15:56:20)
 on OS X 10.7


 when using stats.fisher_exact I am getting errors that make the function
 unusable for cases where I might have a case where both values in a column
 or a row are 0.
 {{{
 o,p =stats.fisher_exact([[0, 1],[0, 2]]) gives
 ValueError                                Traceback (most recent call
 last)

 /Users/aaron/PythonCourse/S9.2_mapping_snps/<ipython console> in
 <module>()

 /Library/Frameworks/EPD64.framework/Versions/7.0/lib/python2.7/site-
 packages/scipy/stats/stats.py in fisher_exact(table)
    2257
    2258     epsilon = 1 - 1e-4
 -> 2259     if float(np.abs(pexact - pmode)) / np.abs(np.max(pexact,
 pmode)) <= 1 - epsilon:
    2260         return odssratio, 1
    2261

 /Library/Frameworks/EPD64.framework/Versions/7.0/lib/python2.7/site-
 packages/numpy/core/fromnumeric.py in amax(a, axis, out)
    1798     except AttributeError:
    1799         return _wrapit(a, 'max', axis, out)
 -> 1800     return amax(axis, out)
    1801
    1802

 ValueError: cannot convert float NaN to integer

 and I get a different ValueError in other cases:

 o,p =stats.fisher_exact([[10, 0],[10, 0]])
 ---------------------------------------------------------------------------
 ValueError                                Traceback (most recent call
 last)

 /Users/aaron/PythonCourse/S9.2_mapping_snps/<ipython console> in
 <module>()

 /Library/Frameworks/EPD64.framework/Versions/7.0/lib/python2.7/site-
 packages/scipy/stats/stats.py in fisher_exact(table)
    2257
    2258     epsilon = 1 - 1e-4
 -> 2259     if float(np.abs(pexact - pmode)) / np.abs(np.max(pexact,
 pmode)) <= 1 - epsilon:
    2260         return odssratio, 1
    2261

 /Library/Frameworks/EPD64.framework/Versions/7.0/lib/python2.7/site-
 packages/numpy/core/fromnumeric.py in amax(a, axis, out)
    1798     except AttributeError:
    1799         return _wrapit(a, 'max', axis, out)
 -> 1800     return amax(axis, out)
    1801
    1802

 ValueError: axis(=1) out of bounds
 }}}

--

Comment(by rgommers):

 Not a very informative error indeed. The question is how to handle this.
 The column of zeros can be sampling or structural zeros. In the latter
 case this test is not valid. In the former case, one solution I've seen is
 to replace 0 by 0.5 and then calculate like normal.

 What would you expect the function to do?

-- 
Ticket URL: <http://projects.scipy.org/scipy/ticket/1489#comment:1>
SciPy <http://www.scipy.org>
SciPy is open-source software for mathematics, science, and engineering.


More information about the Scipy-tickets mailing list