[IPython-user] Missing Modules on Ipengines

[Hao Xiong]

Hi,

I am using ipengine in IPython to run multiple instances of a program.
The program follows map-reduce pattern where it processes an input and
writes results to files within its own directory. Right now I am testing it
on one machine with two engines and I get very weired errors.

The program is split into multiple files, one of which, "cluster.py", 
create
a MultiEngineClient and calls an imported function with map method.
The imported function, "main," is in "para.py." There is also a class 
called "Options" in
"para.py" file, and whose constructor is called inside the imported 
function. The error I get
complains about undefined "Options" variable. What is even more odd is that
when I tried to add a print statement to sys.stderr (useless, I know), 
the exception
becomes unknown variable sys, even though sys is imported at the top of 
"para.py."
Adding an import statement inside "main" makes the print statement good.

I think this has to do with ipengine's environment, its path, but I 
cannot be sure.
I tried to make a simple test case using two separate files but the 
simple case works.
So right now I am at a loss. I would like some help on my problem and I 
have some
general questions about the inner working of ipengine.

1. Is there a way to connect pdb to ipengine in order to inspect the stack.
2. What exactly is the environment of ipengine? I am concerned with working
     directory, path, PYTHONPATH, site modules. This is really import when
     a program is split into multiple files, needs calling external 
programs, and
     when outputs go to files.
3. What happens when ipengine needs to call an external program through 
subprocess
     module? What about relative path to regular files?

Thanks for any help.

Regards,
Hao