[IPython-dev] DAG Dependencies

MinRK benjaminrk@gmail....
Thu Oct 28 02:57:34 CDT 2010


Hello,

In order to test/demonstrate arbitrary DAG dependency support in the new ZMQ
Python scheduler, I wrote an example using NetworkX, as Fernando suggested.

It generates a random DAG with a given number of nodes and edges, runs a set
of empty jobs (one for each node) using the DAG as a dependency graph, where
each edge represents a job depending on another.
It then validates the results, ensuring that no job ran before its
dependencies, and draws the graph, with nodes arranged in X according to
time, which means that all arrows must point to the right if the
time-dependencies were met.

It happily handles pretty elaborate (hundreds of edges) graphs.

Too bad I didn't have this done for today's Py4Science talk.

Script can be found here:
http://github.com/minrk/ipython/blob/newparallel/examples/demo/dagdeps.py

-MinRK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-dev/attachments/20101028/b0b21795/attachment-0001.html 


More information about the IPython-dev mailing list