[IPython-dev] Musings: syntax for high-level expression of parallel (and other) execution control

Edward K. Ream edreamleo@gmail....
Fri Sep 4 07:01:51 CDT 2009

On Fri, Sep 4, 2009 at 3:31 AM, Fernando Perez <fperez.net@gmail.com> wrote:

> The code below shows an implementation of a simple for  loop directly
> and via a decorator.  Both versions do the same thing, but the point
> is that by providing such decorators, we can *trivially* provide a
> GCD-style API for users to express their parallelism and have
> execution chunks handled by ipython remotely.

Fascinating.  Let me ask some basic questions to see if I understand.

1.  Both loops do:  results = [None]*count

Is synchronization needed to update this array?

2. You call the decorator @for_each.  Would @parallel be more descriptive?

3. The docstring for for_each is:

   """This decorator-based loop does a normal serial run.
   But in principle it could be doing the dispatch remotely,
   or into a thread pool, etc.

So you are thinking that a call might be something like:

    def call(func):
        for i in iterable:
            << create thread >>

And these threads would place their computations in the global results?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-dev/attachments/20090904/045b42ba/attachment.html 

More information about the IPython-dev mailing list