[IPython-User] ipython parallelism and closures

[Andrew Jaffe]

Dear all,

I have what seems to be a very simply parallelizable task:

     ghdirs = [ './y/_x_%d/' % isim for isim in range(*nsim) ]
     gh = map(lambda ghd:
                     GHcalc.getGH(freq=freq, num=num, nc=nc, GHdir=ghd,
                        verbose=False, no_offset_correction=True),
              ghdirs)

where the variables freq, num and nc are defined prior to this block.

Because of the presence of these variables, this is a closure, which is 
unpicklable, and therefore cannot be sent to ipython clients, hence I 
get the message "ValueError: Sorry, cannot pickle code objects with 
closures" when I replace map with dview.map_sync.

I am sure there must be a trivial way to parallize this... Can anyone help?

Andrew