[IPython-User] IPython parallel and namedtuple

[John Reid]

Hi,

I think the parallel functionality in ipython used to work with
collections.namedtuples but does not with version 0.12.1.

For example if I have module parallel_helper.py:

from collections import namedtuple
Result = namedtuple('Result', 'pid duration')

and a script:

def sleep(secs):
    import os, time, parallel_helper
    start = time.time()
    time.sleep(secs)
    return parallel_helper.Result(os.getpid(), time.time() - start)

rc = parallel.Client()
v = rc.load_balanced_view()
async_result = v.map_async(sleep, range(3, 0, -1), ordered=False)
for ar in async_result:
    print ar

then an error is raised when the results are passed back:

RemoteError: TypeError(__new__() takes exactly 3 arguments (2 given))
Traceback (most recent call last):
  File
"/usr/lib/python2.7/dist-packages/IPython/parallel/engine/streamkernel.py",
line 345, in apply_request
    packed_result,buf = serialize_object(result)
  File "/usr/lib/python2.7/dist-packages/IPython/parallel/util.py", line
230, in serialize_object
    clist = canSequence(obj)
  File "/usr/lib/python2.7/dist-packages/IPython/utils/pickleutil.py",
line 121, in canSequence
    return t([can(i) for i in obj])
  File "/usr/lib/python2.7/dist-packages/IPython/utils/pickleutil.py",
line 105, in can
    return canSequence(obj)
  File "/usr/lib/python2.7/dist-packages/IPython/utils/pickleutil.py",
line 121, in canSequence
    return t([can(i) for i in obj])
TypeError: __new__() takes exactly 3 arguments (2 given)

I think the problem is somewhere in the code that treats lists and
tuples as special cases in pickleutil.py:

def canSequence(obj):
    if isinstance(obj, (list, tuple)):
        t = type(obj)
        return t([can(i) for i in obj])
    else:
        return obj

As far as I can remember namedtuples that were well defined in their own
modules were perfectly usable across the parallel interface.

Is there any way that I can still use them? I find them extremely useful.

Thanks,
John.