[IPython-dev] Load Balanced PyZMQ multikernel example
Mon Jul 19 22:18:04 CDT 2010
This is a very nice demonstration of what you can do by hooking up
some 0mq sockets and devices. I find that the possibilities are so
many that that it takes a while to really let it sink in.
On Mon, Jul 19, 2010 at 5:33 PM, MinRK <email@example.com> wrote:
> I thought this might be of some interest to the zmq related IPython folks:
> pyzmq has a basic multiple client-one kernel remote process example called
> 'kernel'. This morning, to explore zmq devices, I wrote a derived example
> that is multiple client - multiple kernel, and load balanced across kernels,
> and called it 'multikernel'. It took about an hour.
> The code is trivial, and uses the zmq XREQ socket's round robin load
> o The main addition is a relay process containing two zmq devices: a queue
> device for the XREQ/XREP connection, and a forwarder for PUB/SUB.
> o kernel.py had to change a little, since two socket IDs are contained in
> each message instead of just one, and its sockets connect instead of bind.
> o frontend.py and other code didn't have to change a letter.
> o Exactly zero work is done in Python in the relay process after the
> creation of the ØMQ devices.
> It does have some weird behavior, since even the tab-completion requests are
> load balanced, so if you have two kernels, and you do:
> each press of the tab key will produce different results - which is fun to
> watch, if not especially useful.
> I even did a quick and dirty screencast to show 30 seconds of using it with
> 2 clients and 2 kernels.
> The example is pushed to my pyzmq fork on github, and depends on that fork
> for its implementation of ØMQ devices, not yet merged into Brian's trunk.
> ØMQ really is spiffy.
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
More information about the IPython-dev