[IPython-User] Multiprocessing question

MinRK benjaminrk@gmail....
Wed Feb 1 12:05:33 CST 2012


2012/2/1 Ricardo Amézquita <ramezquitao@gmail.com>

> I have some code that uses multiprocessing to parallelize some
> calculation. I know that previous versions of Ipython had some issues with
> multiprocessing. Yesterday, I ran some long time (3 h) calculation using my
> parallel code and Ipython. In the standard console the code ran without
> problems, but in the web notebook, and the qtconsole, the program never
> finished. The code didn't break, so there was no error message, it just
> simply sat there.
> So my questions are:
> - are there still issues between multiprocessing and Ipython?
>

The issue was most directly with pickling of interactively defined
classes/objects, which multiprocessing uses.  We believe this to be fixed,
but it may not be in all cases.


> - My software will be greatly benefited if you could run it  from the
> Ipython qtconsole, or web notebook. If I want to keep the parallel
> processing would you say it is better to use Ipython's parallel
> infrastructure instead multiprocessing (or others)?
>

It's fine to use multiprocessing if that's what you are already using.  Can
you provide some more information, and ideally actual code samples that
fail in IPython and not in regular Python?

-MinRK


>
> Thanks
> Ricardo Amézquita Orozco
>
> ramezquitao@cihologramas.com
> ramezquitao@unal.edu.co
> ramezquitao@gmail.com
>
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120201/67eb3304/attachment.html 


More information about the IPython-User mailing list