[Numpy-discussion] Release blockers for 1.4.0 ?
Bruce Southey
bsouthey@gmail....
Tue Dec 8 09:10:51 CST 2009
On 12/08/2009 08:36 AM, Charles R Harris wrote:
>
>
> On Tue, Dec 8, 2009 at 12:31 AM, David Cournapeau
> <david@ar.media.kyoto-u.ac.jp <mailto:david@ar.media.kyoto-u.ac.jp>>
> wrote:
>
> Pierre GM wrote:
> > A bit of background first;
> > In the first implementations of numpy.core.ma
> <http://numpy.core.ma>, the approach was to get rid of the data
> that could cause problem beforehand by replacing them with safe
> values. Turned out that finding these data is not always obvious
> (cf the problem of defining a domain when dealing with exponents
> that we discussed a while back on this list), and that all in all,
> it is faster to compute first and deal with the problems
> afterwards. Of course, the user doesn't need the warnings if
> something goes wrong, so disabling them globally looked like a way
> to go. I thought the disabling would be only for numpy.ma
> <http://numpy.ma>, though
>
> I don't think there is an easy (or any) way to disable this at module
> level. Please be sure to always do so in a try/finally (like you would
> handle a file object to guarantee it is always closed, for example),
> because otherwise, test failures and SIGINT (ctrl+C) will pollute the
> user environment.
>
> >
> > Anyhow, running numpy.ma.test() on 1.5.x, I get 36 warnings by
> getting rid of the np.seterr(all='ignore') in numpy.ma.core. I can
> go down to 2 by saving the seterr state before the computations in
> _Masked/DomainUnary/BinaryOperation and restoring it after the
> computation. I gonna try to find where the 2 missing warnings come
> from.
> > The 281 tests + 36 warnings take 4.087s to run, the 2 warning
> version 5.95s (but I didn't try to go to much into timing details...)
>
> Setting/unsetting the FPU state definitely has a cost. I don't
> know how
> significant it would be for your precise case, though: is the cost
> because of setting/unsetting the state in the test themselves or ? We
> may be able to improve the situation later on once we have better
> numbers.
>
> A proper solution to this FPU exception may require hard work (because
> of the inherent asynchronous nature of signals, because signals behave
> very differently on different platforms, and because I don't think we
> can afford spending too many cycles on it).
>
> >
> > So, what do you want me to do guys ? Commit the fixes to the
> trunk ? Backporting them to 1.4.x ?
>
> I have already committed the removal of the global np.seterr in the
> trunk. I feel like backporting this one to 1.4.x is a good idea
> (because
> it could be considered as a regression), but maybe someone has a
> strong
> case against it.
>
>
> At this point it isn't a regression, it is a tradition. I think it
> best to leave the fix out of 1.4 and make the change for 1.5 because
> it is likely to break user code.
>
> Chuck
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
I understand the reason for the masked arrays behavior but changing the
seterr default will be a problem until ma is changed. With Python 2.6
and numpy '1.4.0.dev7750', the current default works but fails when
changing the seterr default.
>>> a = np.ma.masked_array([-1, 0, 1, 2, 3], mask=[0, 0, 0, 0, 1])
>>> np.sqrt(a)
masked_array(data = [-- 0.0 1.0 1.41421356237 --],
mask = [ True False False False True],
fill_value = 999999)
>>> np.seterr(all='raise')
{'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under':
'ignore'}
>>> np.sqrt(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
FloatingPointError: invalid value encountered in sqrt
Furthermore, with np.seterr(all='raise') scipy.test() stops with an
numpy error (output below) and scipy also appears to have some test
errors. Should I submit a bug report for that error?
Also, with seterr(all='raise') there are 25 test errors in numpy 1.3.0
and 25 test errors in numpy 1.4.0.dev7750 (only 10 are the same). So
these tests would need to be resolved.
So I agree with Chuck that we need to hold off the change in default
until after 1.4 because there may be other code that has similar
behavior to ma and scipy. Also we need both numpy and scipy to pass all
the tests with this default.
Bruce
>>> sp.__version__
'0.8.0.dev6119'
>>> sp.test()
Running unit tests for scipy
NumPy version 1.4.0.dev7750
NumPy is installed in /usr/lib64/python2.6/site-packages/numpy
SciPy version 0.8.0.dev6119
SciPy is installed in /usr/lib64/python2.6/site-packages/scipy
Python version 2.6 (r26:66714, Jun 8 2009, 16:07:29) [GCC 4.4.0
20090506 (Red Hat 4.4.0-4)]
nose version 0.10.4
................................................................................................................................................................................................F.....................................................................................................................K..K............................................................EEE........E...................................................F......................................................................................................................................................................................Warning:
1000000 bytes requested, 20 bytes read.
............................................Exception AttributeError:
"'netcdf_file' object has no attribute 'mode'" in <bound method
netcdf_file.close of <scipy.io.netcdf.netcdf_file object at 0x34b8f50>>
ignored
........................[ 8 103 111 24 47] [ 8 103 111 24 47]
.[[ 77 53]
{snip}
[ 65 69 70 80 46]]
...............................................................................................................................................SSSSSS......SSSSSS......SSSS.....................................................................E..E...........................................................................................................................................................................................................E...............................................................................................................................................Result
may be inaccurate, approximate err = 1.18043920589e-08
...Result may be inaccurate, approximate err = 4.28420877755e-10
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................E.SSSSSSSSSSS..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................E.E..............................E...........K.K...........................................................................................................................................................................................E.......................................................................E...................E...........................................EEEE....................EE....EEEE......EEKEEEE....K.......E.SSSSSE................E..E..........................................................................Traceback
(most recent call last):
File "<stdin>", line 1, in <module>
File
"/usr/lib64/python2.6/site-packages/numpy/testing/nosetester.py", line
335, in test
t = NumpyTestProgram(argv=argv, exit=False, plugins=plugins)
File "/usr/lib/python2.6/site-packages/nose/core.py", line 219, in
__init__
argv=argv, testRunner=testRunner, testLoader=testLoader)
File "/usr/lib64/python2.6/unittest.py", line 817, in __init__
self.runTests()
File
"/usr/lib64/python2.6/site-packages/numpy/testing/noseclasses.py", line
300, in runTests
self.result = self.testRunner.run(self.test)
File "/usr/lib/python2.6/site-packages/nose/core.py", line 62, in run
test(result)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in
__call__
return self.run(*arg, **kw)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 168, in run
test(orig)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in
__call__
return self.run(*arg, **kw)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 168, in run
test(orig)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in
__call__
return self.run(*arg, **kw)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 168, in run
test(orig)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in
__call__
return self.run(*arg, **kw)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 168, in run
test(orig)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in
__call__
return self.run(*arg, **kw)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 168, in run
test(orig)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 138, in
__call__
return self.run(*arg, **kw)
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 161, in run
for test in self._tests:
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 283, in
_get_wrapped_tests
for test in self._get_tests():
File "/usr/lib/python2.6/site-packages/nose/suite.py", line 79, in
_get_tests
for test in self.test_generator:
File "/usr/lib/python2.6/site-packages/nose/loader.py", line 221, in
generate
for test in g():
File
"/usr/lib64/python2.6/site-packages/scipy/stats/tests/test_continuous_basic.py",
line 168, in test_cont_basic
m,v = distfn.stats(*arg)
File
"/usr/lib64/python2.6/site-packages/scipy/stats/distributions.py", line
836, in stats
mu = self._munp(1.0,*goodargs)
File
"/usr/lib64/python2.6/site-packages/scipy/stats/distributions.py", line
563, in _munp
return self.generic_moment(n,*args)
File "/usr/lib64/python2.6/site-packages/numpy/lib/function_base.py",
line 1823, in __call__
_res = array(self.ufunc(*newargs),copy=False,
FloatingPointError: underflow encountered in _mom1_sc (vectorized)
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20091208/a91e2df9/attachment-0001.html
More information about the NumPy-Discussion
mailing list