[IPython-user] Missing Modules on Ipengines

Hao Xiong hao@biostat.ucsf....
Fri Feb 20 11:20:14 CST 2009


Hi Vishal,

Thanks for the reply. Actually I am a little confused about
this remote execution, so if you could explain a bit about it,
it would be great.

The thing is Python has static scoping, so reference to random
should have been embedded inside rfcn. To further highlight this,
I changed the import code to

import caller
mec.map(callee.rfcn, [1,1])

and error persists. This time the whole module is imported,
which includes "import random" statement.

For my real program, moving import statement inside the function
is impractical because it looks like this:

class Options():
    ...

def main():
    opt = Options()
    ...

The function main is then called through map method of
MultiEngineClient. Besides there are other files eventually
called by and through main,  so the problem exists on a large scale.

Thanks for you help.

Best,
Hao


Vishal Vatsa wrote:
> Hi Hao,
>
> I think I see your problem,
>
> You are in effect sending the rfcn method for remote execution but it
> has a reference
> to random which has not been imported on the ipengine.
> Also it might help to think in terms of, you are packaging every thing
> needed to execute
> on the remote node.
>
> So if you just redo the rfcn method as:
> def rfcn(seed):
>    import random
>    random.seed(seed)
>    return random.random()
>
>
> it should be fine.
> Hope that makes sense, if not give me a shout and I will try to explain further.
>
> Regards,
> -vishal
>
> 2009/2/20 Hao Xiong <hao@biostat.ucsf.edu>:
>   
>> Hi Brian,
>>
>> Attached are three files, which show the behaviors I encountered.
>>
>> caller.py uses MultiEngineClient's map method to generate random
>> numbers in parallel; the actual generating function is rfcn in callee.py.
>> The random module is imported within callee.py at the file level.
>> The exception raised is that "'random' is not defined." Moving the import
>> statement inside function rfcn makes everything work.
>> The log file is the traceback.
>>
>> Regards,
>> Hao
>>
>>
>>     



More information about the IPython-user mailing list