[IPython-user] Missing Modules on Ipengines
Thu Feb 19 13:13:15 CST 2009
I am using ipengine in IPython to run multiple instances of a program.
The program follows map-reduce pattern where it processes an input and
writes results to files within its own directory. Right now I am testing it
on one machine with two engines and I get very weired errors.
The program is split into multiple files, one of which, "cluster.py",
a MultiEngineClient and calls an imported function with map method.
The imported function, "main," is in "para.py." There is also a class
called "Options" in
"para.py" file, and whose constructor is called inside the imported
function. The error I get
complains about undefined "Options" variable. What is even more odd is that
when I tried to add a print statement to sys.stderr (useless, I know),
becomes unknown variable sys, even though sys is imported at the top of
Adding an import statement inside "main" makes the print statement good.
I think this has to do with ipengine's environment, its path, but I
cannot be sure.
I tried to make a simple test case using two separate files but the
simple case works.
So right now I am at a loss. I would like some help on my problem and I
general questions about the inner working of ipengine.
1. Is there a way to connect pdb to ipengine in order to inspect the stack.
2. What exactly is the environment of ipengine? I am concerned with working
directory, path, PYTHONPATH, site modules. This is really import when
a program is split into multiple files, needs calling external
when outputs go to files.
3. What happens when ipengine needs to call an external program through
module? What about relative path to regular files?
Thanks for any help.
More information about the IPython-user