[IPython-dev] Load Balanced PyZMQ multikernel example

MinRK benjaminrk@gmail....
Mon Jul 19 19:33:03 CDT 2010

I thought this might be of some interest to the zmq related IPython folks:

pyzmq has a basic multiple client-one kernel remote process example called
'kernel'.  This morning, to explore zmq devices, I wrote a derived example
that is multiple client - multiple kernel, and load balanced across kernels,
and called it 'multikernel'. It took about an hour.

The code is trivial, and uses the zmq XREQ socket's round robin load
o The main addition is a relay process containing two zmq devices: a queue
device for the XREQ/XREP connection, and a forwarder for PUB/SUB.
o kernel.py had to change a little, since two socket IDs are contained in
each message instead of just one, and its sockets connect instead of bind.
o frontend.py and other code didn't have to change a letter.
o Exactly zero work is done in Python in the relay process after the
creation of the ØMQ devices.

It does have some weird behavior, since even the tab-completion requests are
load balanced, so if you have two kernels, and you do:
each press of the tab key will produce different results - which is fun to
watch, if not especially useful.

I even did a quick and dirty screencast to show 30 seconds of using it with
2 clients and 2 kernels.

The example is pushed to my pyzmq fork on github, and depends on that fork
for its implementation of ØMQ devices, not yet merged into Brian's trunk.

ØMQ really is spiffy.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-dev/attachments/20100719/3cf8aee5/attachment.html 

More information about the IPython-dev mailing list