[IPython-user] Some comments/questions on TaskClient

[Gael Varoquaux]

Hey,

I am using IPython1 to distribute some embarrassingly parallel problems
on my different CPUs. I have some questions/comments.

First of all, I couldn't figure out how to do push/pulls of variables
with the Taskclient. I understand this might not seem something you would
want to do with a task-based client, but here is a good use case:

    I want to map a function f to a set of parameters s. The function
    knows how to perform all the work, but I have a few constants I want
    to pass to the function, for instance the name of an output
    directory. I can of course pass this as a parameter to my function,
    but this imposes generating a list of this parameter. This is
    slightly ugly.

Second question: I get crashes in my engines, probably because of a
MemoryError, how ever I am unable to find where the traceback is stored.
How can debug these crashes?

Thanks for your work on IPython1.

Gaël