[IPython-User] parallel questions

MinRK benjaminrk@gmail....
Thu Feb 2 14:24:17 CST 2012


On Thu, Feb 2, 2012 at 10:36, Toby Burnett <tburnett@uw.edu> wrote:

>  I’m a heavy user of the IPython 0.12 parallel, and have gotten some very
> useful help from Min.
>

Yay! I'm glad it is useful.



> ****
>
> Something I have not figured out how to do, however: If an engine fails,
> its GUID is reported, but, having used a balanced view to assign tasks, I
> don’t know which task it was assigned, which would be very useful for
> debugging.
>

Hm, this is definitely trickier than it should be.  If you try to block/get
your AsyncResults, it will tell you what tasks failed by raising
EngineError with info about the task and engine.  But it should be easier
to find this out.  Currently, the best way is probably to use db_query:

rc.db_query(dict(engine_uuid=<engine_id>), keys=['msg_id', 'completed',
'result_content'])

>From which you can see which one is the first to fail with this error.

I will look into a better way to diagnose this kind of thing.


> ****
>
> ** **
>
> Another vague request is for tools to set up, monitor, and kill a cluster.
> It occurs to me that the notebook interface is ideal for both understanding
> how to do things, and easily customizing for an individual setup: in my
> case the notebook server runs in the same environment as the engines, so a
> notebook is a nice way to perform cluster management.
>

Good thing Brian is working on exactly that today :)


> ****
>
> ** **
>
> --Toby Burnett****
>
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120202/de0a2810/attachment.html 


More information about the IPython-User mailing list