[IPython-user] Ipython 1 parallel execution on a multicore machine

Brian Granger ellisonbg.net@gmail....
Tue Mar 4 09:28:31 CST 2008


Markus,

> I have a question regarding ipython1's parallel execution. I have not used
> it yet, but it seems like an easy way to parallelize existing python code.
> In my case, I would like to parallelize onto different CPU cores inside the
> same machine. As my datasets are rather big, I would prefer if the data was
> not copied to be operated on by a different core. Is IPython capable of
> handling this?

Ipython1 is based on processes, not threads, so you won't be able to
use shared memory.  Furthermore, you won't be able to easily use
Python for this because of the global interpreter lock (GIL).  In
fact, the GIL is a major reason ipython1 is based on processes, not
threads.  Until Python itself gets rid of the GIL, we will have this
limitation.  The only exceptions to this are:

1) Use Jython or IronPython which don't have the GIL
2) Move the threading code down to C/C++ code that doesn't use the
Python C API and thus can release the  GIL

But, in these 3 cases, IPython1 won't be of much use.  But, if you can
get by using processes, IPython1 would be a great solution.

Brian

> Thanks,
>
> Markus
> _______________________________________________
>  IPython-user mailing list
>  IPython-user@scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-user
>
>


More information about the IPython-user mailing list