[IPython-user] processing.Pool in ipython

[Robin]

Hi,

I have a problem using the processing module (pyprocessing) from
within ipython.

Using the following simple file (taken from the examples):
pooltest.py
--------------
from processing import Pool

def f(x):
    return x*x

pool = Pool(processes=4)              # start 4 worker processes
print pool.map(f, range(10))          # prints "[0, 1, 4,..., 81]"
---------------

I get the following output:
with python it works
robin-mbp:handy robince$ python pooltest.py
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

With ipython it doesn't (see below). Is it know that this doesn't work
with ipython? I was hoping to use it interactively to try and produce
a multiprocess version of the handythread.py functions on the scipy
wiki but if it doesn't work with ipython I think I'll give up on it.

Thanks for any help,

Robin

robin-mbp:handy robince$ ipython pooltest.py
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/threading.py",
line 460, in __bootstrap
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/threading.py",
line 440, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/pool.py",
line 207, in _handleTasks
    put(task)
PicklingError: Can't pickle <type 'function'>: attribute lookup
__builtin__.function failed

^CProcess PoolWorker-4:
Process PoolWorker-2:
Traceback (most recent call last):
Traceback (most recent call last):
Process PoolWorker-1:  File
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
line 227, in _bootstrap

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
line 227, in _bootstrap
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
line 85, in run
    self._target(*self._args, **self._kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/pool.py",
line 54, in worker
    for job, i, func, args, kwds in iter(inqueue.get, None):
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/queue.py",
line 325, in get
    racquire()
    self.run()
KeyboardInterrupt
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
line 85, in run
    self._target(*self._args, **self._kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/pool.py",
line 54, in worker
    for job, i, func, args, kwds in iter(inqueue.get, None):
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/queue.py",
line 327, in get
    return recv()
KeyboardInterrupt
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
line 227, in _bootstrap
Process PoolWorker-3:
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
line 85, in run
    self._target(*self._args, **self._kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/pool.py",
line 54, in worker
Traceback (most recent call last):
    for job, i, func, args, kwds in iter(inqueue.get, None):
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/queue.py",
line 325, in get
    racquire()
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
line 227, in _bootstrap
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/process.py",
line 85, in run
    self._target(*self._args, **self._kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/pool.py",
line 54, in worker
    for job, i, func, args, kwds in iter(inqueue.get, None):
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/processing-0.52-py2.5-macosx-10.3-fat.egg/processing/queue.py",
line 325, in get
    racquire()