[Numpy-discussion] Release blockers for 1.4.0 ?

Bruce Southey bsouthey@gmail....
Tue Dec 8 09:10:51 CST 2009


On 12/08/2009 08:36 AM, Charles R Harris wrote:
>
>
> On Tue, Dec 8, 2009 at 12:31 AM, David Cournapeau 
> <david@ar.media.kyoto-u.ac.jp <mailto:david@ar.media.kyoto-u.ac.jp>> 
> wrote:
>
>     Pierre GM wrote:
>     > A bit of background first;
>     > In the first implementations of numpy.core.ma
>     <http://numpy.core.ma>, the approach was to get rid of the data
>     that could cause problem beforehand by replacing them with safe
>     values. Turned out that finding these data is not always obvious
>     (cf the problem of defining a domain when dealing with exponents
>     that we discussed a while back on this list), and that all in all,
>     it is faster to compute first and deal with the problems
>     afterwards. Of course, the user doesn't need the warnings if
>     something goes wrong, so disabling them globally looked like a way
>     to go. I thought the disabling would be only for numpy.ma
>     <http://numpy.ma>, though
>
>     I don't think there is an easy (or any) way to disable this at module
>     level. Please be sure to always do so in a try/finally (like you would
>     handle a file object to guarantee it is always closed, for example),
>     because otherwise, test failures and SIGINT (ctrl+C) will pollute the
>     user environment.
>
>     >
>     > Anyhow, running numpy.ma.test() on 1.5.x, I get 36 warnings by
>     getting rid of the np.seterr(all='ignore') in numpy.ma.core. I can
>     go down to 2 by saving the seterr state before the computations in
>     _Masked/DomainUnary/BinaryOperation and restoring it after the
>     computation. I gonna try to find where the 2 missing warnings come
>     from.
>     > The 281 tests + 36 warnings take 4.087s to run, the 2 warning
>     version 5.95s (but I didn't try to go to much into timing details...)
>
>     Setting/unsetting the FPU state definitely has a cost. I don't
>     know how
>     significant it would be for your precise case, though: is the cost
>     because of setting/unsetting the state in the test themselves or ? We
>     may be able to improve the situation later on once we have better
>     numbers.
>
>     A proper solution to this FPU exception may require hard work (because
>     of the inherent asynchronous nature of signals, because signals behave
>     very differently on different platforms, and because I don't think we
>     can afford spending too many cycles on it).
>
>     >
>     > So, what do you want me to do guys ? Commit the fixes to the
>     trunk ? Backporting them to 1.4.x ?
>
>     I have already committed the removal of the global np.seterr in the
>     trunk. I feel like backporting this one to 1.4.x is a good idea
>     (because
>     it could be considered as a regression), but maybe someone has a
>     strong
>     case against it.
>
>
> At this point it isn't a regression, it is a tradition. I think it 
> best to leave the fix out of 1.4 and make the change for 1.5 because 
> it is likely to break user code.
>
> Chuck
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>    

I understand the reason for the masked arrays behavior but changing the 
seterr default will be a problem until ma is changed. With Python 2.6 
and numpy '1.4.0.dev7750', the current default works but fails when 
changing the seterr default.
 >>> a = np.ma.masked_array([-1, 0, 1, 2, 3], mask=[0, 0, 0, 0, 1])
 >>> np.sqrt(a)
masked_array(data = [-- 0.0 1.0 1.41421356237 --],
              mask = [ True False False False  True],
        fill_value = 999999)
 >>> np.seterr(all='raise')
{'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 
'ignore'}
 >>> np.sqrt(a)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
FloatingPointError: invalid value encountered in sqrt

Furthermore, with np.seterr(all='raise') scipy.test() stops with an 
numpy error (output below) and scipy also appears to have some test 
errors. Should I submit a bug report for that error?

Also, with seterr(all='raise') there are 25 test errors in numpy 1.3.0 
and 25 test errors in numpy 1.4.0.dev7750 (only 10 are the same). So 
these tests would need to be resolved.

So I agree with Chuck that we need to hold off the change in default 
until after 1.4 because there may be other code that has similar 
behavior to ma and scipy. Also we need both numpy and scipy to pass all 
the tests with this default.

Bruce


 >>> sp.__version__
'0.8.0.dev6119'
 >>> sp.test()
Running unit tests for scipy
NumPy version 1.4.0.dev7750
NumPy is installed in /usr/lib64/python2.6/site-packages/numpy
SciPy version 0.8.0.dev6119
SciPy is installed in /usr/lib64/python2.6/site-packages/scipy
Python version 2.6 (r26:66714, Jun  8 2009, 16:07:29) [GCC 4.4.0 
20090506 (Red Hat 4.4.0-4)]
nose version 0.10.4
................................................................................................................................................................................................F.....................................................................................................................K..K............................................................EEE........E...................................................F......................................................................................................................................................................................Warning: 
1000000 bytes requested, 20 bytes read.
............................................Exception AttributeError: 
"'netcdf_file' object has no attribute 'mode'" in <bound method 
netcdf_file.close of <scipy.io.netcdf.netcdf_file object at 0x34b8f50>> 
ignored
........................[  8 103 111  24  47] [  8 103 111  24  47]
.[[ 77  53]

{snip}

  [ 65  69  70  80  46]]
...............................................................................................................................................SSSSSS......SSSSSS......SSSS.....................................................................E..E...........................................................................................................................................................................................................E...............................................................................................................................................Result 
may be inaccurate, approximate err = 1.18043920589e-08
...Result may be inaccurate, approximate err = 4.28420877755e-10
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................E.SSSSSSSSSSS..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................E.E..............................E...........K.K...........................................................................................................................................................................................E.......................................................................E...................E...........................................EEEE....................EE....EEEE......EEKEEEE....K.......E.SSSSSE................E..E..........................................................................Traceback 
(most recent call last):
   File "<stdin>", line 1, in <module>
   File 
"/usr/lib64/python2.6/site-packages/numpy/testing/nosetester.py", line 
335, in test
     t = NumpyTestProgram(argv=argv, exit=False, plugins=plugins)
   File "/usr/lib/python2.6/site-packages/nose/core.py", line 219, in 
__init__
     argv=argv, testRunner=testRunner, testLoader=testLoader)
   File "/usr/lib64/python2.6/unittest.py", line 817, in __init__
     self.runTests()
   File 
"/usr/lib64/python2.6/site-packages/numpy/testing/noseclasses.py", line 
300, in runTests
     self.result = self.testRunner.run(self.test)
   File "/usr/lib/python2.6/site-packages/nose/core.py", line 62, in run
     test(result)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in 
__call__
     return self.run(*arg, **kw)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 168, in run
     test(orig)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in 
__call__
     return self.run(*arg, **kw)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 168, in run
     test(orig)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in 
__call__
     return self.run(*arg, **kw)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 168, in run
     test(orig)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in 
__call__
     return self.run(*arg, **kw)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 168, in run
     test(orig)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in 
__call__
     return self.run(*arg, **kw)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 168, in run
     test(orig)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in 
__call__
     return self.run(*arg, **kw)
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 161, in run
     for test in self._tests:
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 283, in 
_get_wrapped_tests
     for test in self._get_tests():
   File "/usr/lib/python2.6/site-packages/nose/suite.py", line 79, in 
_get_tests
     for test in self.test_generator:
   File "/usr/lib/python2.6/site-packages/nose/loader.py", line 221, in 
generate
     for test in g():
   File 
"/usr/lib64/python2.6/site-packages/scipy/stats/tests/test_continuous_basic.py", 
line 168, in test_cont_basic
     m,v = distfn.stats(*arg)
   File 
"/usr/lib64/python2.6/site-packages/scipy/stats/distributions.py", line 
836, in stats
     mu = self._munp(1.0,*goodargs)
   File 
"/usr/lib64/python2.6/site-packages/scipy/stats/distributions.py", line 
563, in _munp
     return self.generic_moment(n,*args)
   File "/usr/lib64/python2.6/site-packages/numpy/lib/function_base.py", 
line 1823, in __call__
     _res = array(self.ufunc(*newargs),copy=False,
FloatingPointError: underflow encountered in _mom1_sc (vectorized)
 >>>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20091208/a91e2df9/attachment-0001.html 


More information about the NumPy-Discussion mailing list