[IPython-User] HOWTO invoke a script in parallel

Brian Granger ellisonbg@gmail....
Tue Dec 21 11:09:08 CST 2010


On Tue, Dec 21, 2010 at 7:37 AM, Stefan Seefeld
<stefan.seefeld@gmail.com> wrote:
> On 12/15/2010 05:21 PM, Stefan Seefeld wrote:
>>
>> On 12/15/2010 05:06 PM, Brian Granger wrote:
>>>
>>> Stefan,
>>>
>>> On Wed, Dec 15, 2010 at 8:13 AM, Stefan Seefeld<seefeld@sympatico.ca>
>>>  wrote:
>>>>
>>>> Hello,
>>>>
>>>> I'm trying to invoke a (Python) script ('fftm.py') from within an
>>>> IPython session, in parallel.
>>>
>>> Can you post this script.  You may be misunderstanding how px and
>>> autopx work?  If you can't post the script, can you describe more of
>>> what you are trying to do?
>>
>> I can certainly post this script (see attached). The script loads a bunch
>> of C++ extension modules, creates some distributed objects, and then invokes
>> some functions on them.
>
> Brian,
>
> your reply may have got lost during the mail server downtime, so allow me to
> follow-up:

Thanks for following up...

> I'm still trying to figure out how to run a script interactively in parallel
> (akin to `mpirun -np N <binary>` on the command line). Is this supported at
> all ? Is there some documentation explaining the semantics of such an
> operation ?

Yes, I think what you want is MultiEngineClient.run:

https://github.com/ipython/ipython/blob/master/IPython/kernel/multiengineclient.py#L825

Either that you you could read the file yourself and then use
MultiEngineClient.execute.

See if either of these work.

Cheers,

Brian

> I tried various things, but always got an error, such as
>
> In [3]: px run script.py
> Parallel execution on engines: all
> ERROR: An unexpected error occurred while tokenizing input
> The following traceback may be corrupted or invalid
> The error message is: ('EOF in multi-line statement', (9, 0))
>
> (And in fact I couldn't even read the colorized traceback, since more than
> half of the text appears to be white-on-white.)
>
> Note that pasting the content of the script, line-by-line, does work
> correctly, even with autopx turned on.
>
>
> Any help is greatly appreciated.
>
> Thanks,
>        Stefan
>
>
> --
>
>      ...ich hab' noch einen Koffer in Berlin...
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu
ellisonbg@gmail.com


More information about the IPython-User mailing list