[IPython-dev] Musings: syntax for high-level expression of parallel (and other) execution control

Brian Granger ellisonbg.net@gmail....
Tue Sep 8 22:09:25 CDT 2009


On Tue, Sep 8, 2009 at 1:34 PM, Fernando Perez <fperez.net@gmail.com> wrote:

> On Tue, Sep 8, 2009 at 10:23 AM, Brian Granger <ellisonbg.net@gmail.com>
> wrote:
> > My only complaint is that is a bit unexpected that this actually declares
> > and *calls* the function!
>
> Yes, it is more than a bit surprising at first :)  In fact, I remember
> that bothered me about @interact when I first saw it, and that was
> partly why I tried to make things work using 'with'.  But it's now
> clear to me that we need  a *real* scope for these ideas, and for now,
> 'def' is the only way to get a scope we have, so that's what we'll be
> using.
>
> For reference, the stuff I was trying to implement from the Ars review
> is below in the original Objective C code, plus my own 'pythonization'
> of it, first in serial mode, then using 'with', then with @decos.
>
> This is the page in the review where John Siracusa explains the new
> Block syntax that Apple introduced to C:
>
> http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/10
>
> I think it's really cool, I do hope it makes its way into the language
> itself.
>
>
Yes, this is a very nice article.  It is very nice to see Apple bringing the
"iPod"
style of thinking (stepping back and really thinking about something) to
parallelism and languages.

def analyzeDocument():
>    """A decorator-based version.
>
>    This could in principle work just fine today.  All one would need to do
>    would be to write Python decorators for Apple GCD.  As long as the code
>    being run released the GIL, it would work fine. """
>
>    @dispatch_async(dispatch_get_global_queue(0, 0))
>    def outer():
>        stats = myDoc.analyze()
>        @dispatch_async(dispatch_get_main_queue())
>        def inner():
>            myModel.setDict(stats)
>            myStatsView.setNeedsDisplay(YES)
>            stats.release()
>

OK, just a few comments before I loose a week or two thinking about this....

* While multiprocessing is interesting, by sticking with the fork model
of getting variables to child execution contexts, it really misses the
greater
possibilities that proper scoping gives you.  With these GCD style
constructs
you can do arbitrarily deep nestings of scopes and carefully control what
variables
from the parent scope are seen by children scopes.  This is exactly like
Cilk and
is a fantastic way of doing things.

* Damn the GIL!  If we didn't have the GIL, we could write a simple version
of
dispatch_async that used a simple thread pool we would be off to the races.

* But, we can implement this on top of IPython.kernel and get many of the
same
things.  The only real downside is the performance hit of using processes
rather
than threads.

Cheers,

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-dev/attachments/20090908/3d4ceec3/attachment.html 


More information about the IPython-dev mailing list