[IPython-user] Ipython and multiprocessing

[Michael Schmuker]

> I had the same problem... see the threads here:
> http://thread.gmane.org/gmane.comp.python.ipython.user/4058
> http://thread.gmane.org/gmane.comp.python.ipython.user/4052
Thanks for the pointer!

> I think it is really a shame because it would be really handy to be
> able to use this interactively - something I would like to do is to be
> able to quickly and interactively set off runs ie
> p = Pool(processes=8)
> p.map(lambda x: somefunc(x,10,other_params), range(10))
>
> but this obviously fails because it can't pickle the function.
The strange thing is that it works within plain python, but not in 
IPython. So the multiprocessing module seems to be able to pickle 
interactively defined functions wuite well, but it seems IPython is 
doing something to Pickle which prevents this mechanism form working.

IMHO multiprocessing would perfectly blend in with IPython's parallel 
computing properties, especially for trivial parallelization on SMP 
machines, because one does not need to start and keep track of an 
ipcluster.

Kind regards,

Michael


-- 
Michael Schmuker
Neuroinformatics
Freie Universität Berlin, Germany
http://www.biomachinelearning.net