[IPython-user] TaskClient doesn't work after importing a module in MultiEngineClient
Mon Nov 17 04:16:58 CST 2008
Thanks for your reply.
Brian Granger wrote:
> I haven't look in detail at exactly what the tests script is doing,
> but here is the problem:
> You are doing something that causes the twisted reactor to be started
> twice. The reactor is started for the controller and engine
> automatically when they start and for the client when you import
> IPython.kernel.client. The problem is that when you execute the
> script in the engine, the engine already has the reactor running, but
> at the top of the script you have the import of IPython.kernel.client,
> which tries to start the reactor again.
You are right, this causes the problem. I've put the import of
if __name__ == '__main__':
in the test script, and it works OK.
But shouldn't it be possible to invoke multiple times the import
operation of Ipython.kernel.client?
> More important question, why are you running the test script on the
> engine anyways? You shouldn't need to. Can you tell us more about
> what you are trying to do? I might have a chance this weekend to look
> over your script and make suggestions as well.
Here is where the importing in the engines arises in my code.
My program is composed of two scripts, e.g. myclient.py and mytask.py
The client script is my main program, and the mytask.py script is the code
that will run on the engines.
I have also my own module with common utilities related to the stuff I'm
MyUtils.py. In utils.py there is also my own custom Cluster class which
some operations with the ipcluster, starting, stopping, killing engines
I import MyUtils both in the client script and in the task script, in
the client to
start and connect to the cluster and in the engines to use some other
functionalities in MyUtils
that are not related to the cluster.
If I want to avoid the problem, then probably I should split MyUtils in
two modules, let's say
MyClusterUtils and MyOtherUtils, and import MyClusterUtils only on the
But sometimes it might be necessary to use one of the engines as a
client connected to another ipcontroller,
and it seems impossible to do it, if one cannot import the client in one
of the engines.
More information about the IPython-user