[IPython-user] processing.Pool in ipython

[Robin]

I forgot to add the version information. I on OS X 10.5 with IPython
0.9.1 on python.org python 2.5.1 with processing module v. 0.52.

Cheers

Robin

On Sat, Oct 25, 2008 at 7:07 PM, Robin <robince@gmail.com> wrote:
> Hi,
>
> I have a problem using the processing module (pyprocessing) from
> within ipython.
>
> Using the following simple file (taken from the examples):
> pooltest.py
> --------------
> from processing import Pool
>
> def f(x):
>    return x*x
>
> pool = Pool(processes=4)              # start 4 worker processes
> print pool.map(f, range(10))          # prints "[0, 1, 4,..., 81]"
> ---------------
>
> I get the following output:
> with python it works
> robin-mbp:handy robince$ python pooltest.py
> [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>
> With ipython it doesn't (see below). Is it know that this doesn't work
> with ipython? I was hoping to use it interactively to try and produce
> a multiprocess version of the handythread.py functions on the scipy
> wiki but if it doesn't work with ipython I think I'll give up on it.
>
> Thanks for any help,
>
> Robin
>
> robin-mbp:handy robince$ ipython pooltest.py
> Exception in thread Thread-1:
> Traceback (most recent call last):
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/threading.py",
> line 460, in __bootstrap
>    self.run()
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/threading.py",
> line 440, in run
>    self.__target(*self.__args, **self.__kwargs)
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/pool.py",
> line 207, in _handleTasks
>    put(task)
> PicklingError: Can't pickle <type 'function'>: attribute lookup
> __builtin__.function failed
>
> ^CProcess PoolWorker-4:
> Process PoolWorker-2:
> Traceback (most recent call last):
> Traceback (most recent call last):
> Process PoolWorker-1:  File
> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
> line 227, in _bootstrap
>
> Traceback (most recent call last):
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
> line 227, in _bootstrap
>    self.run()
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
> line 85, in run
>    self._target(*self._args, **self._kwargs)
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/pool.py",
> line 54, in worker
>    for job, i, func, args, kwds in iter(inqueue.get, None):
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/queue.py",
> line 325, in get
>    racquire()
>    self.run()
> KeyboardInterrupt
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
> line 85, in run
>    self._target(*self._args, **self._kwargs)
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/pool.py",
> line 54, in worker
>    for job, i, func, args, kwds in iter(inqueue.get, None):
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/queue.py",
> line 327, in get
>    return recv()
> KeyboardInterrupt
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
> line 227, in _bootstrap
> Process PoolWorker-3:
>    self.run()
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
> line 85, in run
>    self._target(*self._args, **self._kwargs)
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/pool.py",
> line 54, in worker
> Traceback (most recent call last):
>    for job, i, func, args, kwds in iter(inqueue.get, None):
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/queue.py",
> line 325, in get
>    racquire()
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
> line 227, in _bootstrap
>    self.run()
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
> line 85, in run
>    self._target(*self._args, **self._kwargs)
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/pool.py",
> line 54, in worker
>    for job, i, func, args, kwds in iter(inqueue.get, None):
>  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/queue.py",
> line 325, in get
>    racquire()
>