[IPython-User] Parallel direct view pull and bandwidth
Sun Aug 7 21:38:21 CDT 2011
first of all congratulation to all the ipython team for the great work
you did with the release 0.11, and ZMQ 2.1.7. I'm a fan.
I tried the parallel with direct views, that's great.
with a machine A (192.168.1.4)
1) ipcontroller --ip='*'
from machine"A" I remote start ipengine on machine"B" (192.168.1.200)
2) ssh email@example.com ipengine
(I do the point 2) twice to get 2 ipengines, I also tried in local with
then I start ipython to start a client, and I want to evaluate the
bandwith (and latency in a second step).
from IPython.parallel import Client
rc = Client()
dview.execute("FOO=[0.0 for i in xrange(62500)",block=True) # 62500 *
float 64 -> 500kB of data to transfer
T=time.time();tmp=dview.pull('FOO');print time.time() - T # for 2
T=time.time();tmp=dview.pull('FOO',0);print time.time() - T # for 1
in case of machine"A" as controller and machine"B" as 2 ipengines
pull FOO from machine"B" 2 engines 9.03 seconds (2*500kB/9.03 =>
110kB/s) on a network 100Mb/s (12.MB/s)
pull FOO from machine"B" 1 engine 4.7 seconds (500kB/4.7 =>
in case of machine"A" as controller and as 2 ipengines
pull FOO from machine"A" 2 engines 3.4 seconds (2*500kB/3.4 =>
294kB/s) on a local machine
pull FOO from machine"A" 1 engine 2.7 seconds (500kB/2.7 =>
I guess I'm doing something wrong, or I missuse something. Any hint
would be appreciate, anyway I will continue to dig in.
Machine"A" and "B" are running under RHEL5 flavoured distro, with python
2.6, ipython 0.11 installed from source.
Machine"A" is a Quad core 2.6GHz
Machine"B" is an AMD64 3000+ 1.8GHz (pretty old but still alive)
CEO - Electronic and Computer Engineer
More information about the IPython-User