[IPython-dev] zmq/embed_kernel failure in master

MinRK benjaminrk@gmail....
Sun Jun 10 21:57:57 CDT 2012


I don't see any way around issues like this for multiprocess tests.  I've
added tons of shenanigans to try to prevent it, but it simply cannot be
perfect.  There are sleeps everywhere, and some smarter waits, but if you
want the parallel test suite to finish in under 20 minutes, then never
having non-deterministic failures is not achievable.

My answer: don't run the test suite while under heavy load.

-MinRK

On Sun, Jun 10, 2012 at 7:39 PM, Fernando Perez <fperez.net@gmail.com>wrote:

> On Sun, Jun 10, 2012 at 7:36 PM, Fernando Perez <fperez.net@gmail.com>
> wrote:
> > Note: it's not deterministic, but my system was under heavy load at
> > the time when it did occur.  That might give Min some ideas...
>
> These other two also occured only under heavy load, I can't reproduce
> them now that the system isn't being hit so hard:
>
> ======================================================================
> FAIL: %%px --group-outputs=engine
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/home/fperez/usr/lib/python2.7/site-packages/IPython/parallel/tests/test_magics.py",
> line 131, in test_cellpx_groupby_engine
>    self.assertEquals(len(lines), len(expected), io.stdout)
> AssertionError: Parallel execution on engine(s): [1, 2, 3, 4, 5, 6, 7, 8]
> [stdout:1]
> stdout
> stdout2
> [output:1]
> <IPython.core.display.HTML at 0x3c65e50>
> <IPython.core.display.Math at 0x3c65e50>
> Out[1:1]: <IPython.core.display.Math at 0x3c65e50>
> [stdout:2]
> stdout
> stdout2
> [output:2]
> <IPython.core.display.HTML at 0x3afead0>
> <IPython.core.display.Math at 0x3afead0>
> Out[2:1]: <IPython.core.display.Math at 0x3afead0>
> [stdout:3]
> stdout
> stdout2
> [output:3]
> <IPython.core.display.HTML at 0x2dddad0>
> <IPython.core.display.Math at 0x2dddad0>
> Out[3:1]: <IPython.core.display.Math at 0x2dddad0>
> [stdout:4]
> stdout
> stdout2
> [output:4]
> <IPython.core.display.HTML at 0x29c2210>
> <IPython.core.display.Math at 0x29c2210>
> Out[4:1]: <IPython.core.display.Math at 0x29c2210>
> [stdout:5]
> stdout
> stdout2
> [output:5]
> <IPython.core.display.HTML at 0x2749250>
> <IPython.core.display.Math at 0x2749250>
> [stdout:6] stdout
> [output:6]
> <IPython.core.display.HTML at 0x2bf6210>
> [stdout:7]
> stdout
> stdout2
> [output:7]
> <IPython.core.display.HTML at 0x30dded0>
> <IPython.core.display.Math at 0x30dded0>
> Out[7:1]: <IPython.core.display.Math at 0x30dded0>
> [stdout:8]
> stdout
> stdout2
> [output:8]
> <IPython.core.display.HTML at 0x3b1dcd0>
> <IPython.core.display.Math at 0x3b1dcd0>
> Out[8:16]: <IPython.core.display.Math at 0x3b1dcd0>
>
>    '51 != 56' = '%s != %s' % (safe_repr(51), safe_repr(56))
>    u'Parallel execution on engine(s): [1, 2, 3, 4, 5, 6, 7,
> 8]\n[stdout:1] \nstdout\nstdout2\n[output:1]\n<IPython.core.display.HTML
> at 0x3c65e50>\n<IPython.core.display.Math at 0x3c65e50>\nOut[1:1]:
> <IPython.core.display.Math at 0x3c65e50>\n[stdout:2]
> \nstdout\nstdout2\n[output:2]\n<IPython.core.display.HTML at
> 0x3afead0>\n<IPython.core.display.Math at 0x3afead0>\nOut[2:1]:
> <IPython.core.display.Math at 0x3afead0>\n[stdout:3]
> \nstdout\nstdout2\n[output:3]\n<IPython.core.display.HTML at
> 0x2dddad0>\n<IPython.core.display.Math at 0x2dddad0>\nOut[3:1]:
> <IPython.core.display.Math at 0x2dddad0>\n[stdout:4]
> \nstdout\nstdout2\n[output:4]\n<IPython.core.display.HTML at
> 0x29c2210>\n<IPython.core.display.Math at 0x29c2210>\nOut[4:1]:
> <IPython.core.display.Math at 0x29c2210>\n[stdout:5]
> \nstdout\nstdout2\n[output:5]\n<IPython.core.display.HTML at
> 0x2749250>\n<IPython.core.display.Math at 0x2749250>\n[stdout:6]
> stdout\n[output:6]\n<IPython.core.display.HTML at
> 0x2bf6210>\n[stdout:7]
> \nstdout\nstdout2\n[output:7]\n<IPython.core.display.HTML at
> 0x30dded0>\n<IPython.core.display.Math at 0x30dded0>\nOut[7:1]:
> <IPython.core.display.Math at 0x30dded0>\n[stdout:8]
> \nstdout\nstdout2\n[output:8]\n<IPython.core.display.HTML at
> 0x3b1dcd0>\n<IPython.core.display.Math at 0x3b1dcd0>\nOut[8:16]:
> <IPython.core.display.Math at 0x3b1dcd0>\n' =
> self._formatMessage(u'Parallel execution on engine(s): [1, 2, 3, 4, 5,
> 6, 7, 8]\n[stdout:1]
> \nstdout\nstdout2\n[output:1]\n<IPython.core.display.HTML at
> 0x3c65e50>\n<IPython.core.display.Math at 0x3c65e50>\nOut[1:1]:
> <IPython.core.display.Math at 0x3c65e50>\n[stdout:2]
> \nstdout\nstdout2\n[output:2]\n<IPython.core.display.HTML at
> 0x3afead0>\n<IPython.core.display.Math at 0x3afead0>\nOut[2:1]:
> <IPython.core.display.Math at 0x3afead0>\n[stdout:3]
> \nstdout\nstdout2\n[output:3]\n<IPython.core.display.HTML at
> 0x2dddad0>\n<IPython.core.display.Math at 0x2dddad0>\nOut[3:1]:
> <IPython.core.display.Math at 0x2dddad0>\n[stdout:4]
> \nstdout\nstdout2\n[output:4]\n<IPython.core.display.HTML at
> 0x29c2210>\n<IPython.core.display.Math at 0x29c2210>\nOut[4:1]:
> <IPython.core.display.Math at 0x29c2210>\n[stdout:5]
> \nstdout\nstdout2\n[output:5]\n<IPython.core.display.HTML at
> 0x2749250>\n<IPython.core.display.Math at 0x2749250>\n[stdout:6]
> stdout\n[output:6]\n<IPython.core.display.HTML at
> 0x2bf6210>\n[stdout:7]
> \nstdout\nstdout2\n[output:7]\n<IPython.core.display.HTML at
> 0x30dded0>\n<IPython.core.display.Math at 0x30dded0>\nOut[7:1]:
> <IPython.core.display.Math at 0x30dded0>\n[stdout:8]
> \nstdout\nstdout2\n[output:8]\n<IPython.core.display.HTML at
> 0x3b1dcd0>\n<IPython.core.display.Math at 0x3b1dcd0>\nOut[8:16]:
> <IPython.core.display.Math at 0x3b1dcd0>\n', '51 != 56')
> >>  raise self.failureException(u'Parallel execution on engine(s): [1, 2,
> 3, 4, 5, 6, 7, 8]\n[stdout:1]
> \nstdout\nstdout2\n[output:1]\n<IPython.core.display.HTML at
> 0x3c65e50>\n<IPython.core.display.Math at 0x3c65e50>\nOut[1:1]:
> <IPython.core.display.Math at 0x3c65e50>\n[stdout:2]
> \nstdout\nstdout2\n[output:2]\n<IPython.core.display.HTML at
> 0x3afead0>\n<IPython.core.display.Math at 0x3afead0>\nOut[2:1]:
> <IPython.core.display.Math at 0x3afead0>\n[stdout:3]
> \nstdout\nstdout2\n[output:3]\n<IPython.core.display.HTML at
> 0x2dddad0>\n<IPython.core.display.Math at 0x2dddad0>\nOut[3:1]:
> <IPython.core.display.Math at 0x2dddad0>\n[stdout:4]
> \nstdout\nstdout2\n[output:4]\n<IPython.core.display.HTML at
> 0x29c2210>\n<IPython.core.display.Math at 0x29c2210>\nOut[4:1]:
> <IPython.core.display.Math at 0x29c2210>\n[stdout:5]
> \nstdout\nstdout2\n[output:5]\n<IPython.core.display.HTML at
> 0x2749250>\n<IPython.core.display.Math at 0x2749250>\n[stdout:6]
> stdout\n[output:6]\n<IPython.core.display.HTML at 0x2bf6210>\n[stdout:7]
> \nstdout\nstdout2\n[output:7]\n<IPython.core.display.HTML at
> 0x30dded0>\n<IPython.core.display.Math at 0x30dded0>\nOut[7:1]:
> <IPython.core.display.Math at 0x30dded0>\n[stdout:8]
> \nstdout\nstdout2\n[output:8]\n<IPython.core.display.HTML at
> 0x3b1dcd0>\n<IPython.core.display.Math at 0x3b1dcd0>\nOut[8:16]:
> <IPython.core.display.Math at 0x3b1dcd0>\n')
>
>
> ======================================================================
> FAIL: %%px --group-outputs=order
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/home/fperez/usr/lib/python2.7/site-packages/IPython/parallel/tests/test_magics.py",
> line 173, in test_cellpx_groupby_order
>    self.assertEquals(len(lines), len(expected), io.stdout)
> AssertionError: Parallel execution on engine(s): [1, 2, 3, 4, 5, 6, 7, 8]
>
>    '0 != 64' = '%s != %s' % (safe_repr(0), safe_repr(64))
>    'Parallel execution on engine(s): [1, 2, 3, 4, 5, 6, 7, 8]\n' =
> self._formatMessage('Parallel execution on engine(s): [1, 2, 3, 4, 5,
> 6, 7, 8]\n', '0 != 64')
> >>  raise self.failureException('Parallel execution on engine(s): [1, 2,
> 3, 4, 5, 6, 7, 8]\n')
> _______________________________________________
> IPython-dev mailing list
> IPython-dev@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-dev/attachments/20120610/da28e349/attachment.html 


More information about the IPython-dev mailing list