[IPython-dev] Musings: syntax for high-level expression of parallel (and other) execution control

Fernando Perez fperez.net@gmail....
Fri Sep 4 14:30:30 CDT 2009


On Fri, Sep 4, 2009 at 12:17 PM, Darren Dale <dsdale24@gmail.com> wrote:
> Your intent was clear, but the implementation still leaves me
> wondering what is gained by using the @decorator syntax. Maybe I have
> missed something, please bear with me.
>
> With @decorator, you are still passing a function to another function
> to modify its execution, its just a different syntax that achieves the
> same result, isn't it? For example, using your for_each:
> for_each(range(count))(loop) yields the same result without having to
> redefine loop each time you call loop_deco.

I think the only point to take home is that these decorators *execute*
the decorated function right away, and you can ignore it.  Let's look
only at the loop body:

    for i in range(count):
        results[i] = do_work(data, i)

What this simple pattern allows us to do is to (say) profile this
chunk of code right where it is, without having to move it out to
another separate function (where you'd need to create and pass
arguments to provide local variables for example), by doing two
things:

1. Prepend two lines of code:

    @profiled
    def block():

2. Indent by one level the code you wanted to profile, leaving you with:

    @profiled
    def block():
        for i in range(count):
            results[i] = do_work(data, i)

That's it.  All local variables remain available, you don't need to
pass arguments around, you never actually call block() yourself, nada.
 For example, if you wanted the profiling to happen one level deeper
instead, you can do this instead:

    for i in range(count):
        @profiled
        def block():
            results[i] = do_work(data, i)

You've just swapped indentation levels, you didn't need to worry about
passing variables around, and now the profiling happens per-call of
the loop rather than at the top. This may be a silly example, but it
just tries to show how the point of this little exercise is just to
switch scoped execution control with the least intrusive amount of
work possible.

I admit though: this is so trivial, and already in the language, that
maybe I am indeed happy about nothing :)  So I really appreciate the
feedback, especially critical one!  If this is pointless, might as
well kill it before wasting anyone's time with it, so please *do* be
critical of this.

Thanks again,

f


More information about the IPython-dev mailing list