[IPython-User] ipython parallelism returning user-defined types
Wed Nov 10 13:20:01 CST 2010
On 11/10/10 11:33 AM, Brian Granger wrote:
> On Tue, Nov 9, 2010 at 2:18 AM, John Reid <email@example.com
> <mailto:firstname.lastname@example.org>> wrote:
> I'm having some problems returning named tuples from a multi-engine
> client. I do something like:
> from IPython.kernel import client
> mec = client.MultiEngineClient()
> def parallel_get_starts(arg):
> <do some work>
> return <some named tuples>
> results = parallel_get_starts(<some list>)
> but I always get errors like:
> TypeError: __new__() takes exactly 8 arguments (2 given)
> Hmm, that is an odd error that I have never seen. Even the fact that it is a
> TypeError is odd to me. A couple of things to look at:
> * Make sure that named tuples can be pickled and unpickled.
namedtuple objects can be pickled, but you have to be particularly careful.
Creating a namedtuple type at the prompt and then trying to send that over won't
work because the type won't be available on the other side. IPython's FakeModule
will cause more complications. Creating a namedtuple type inside of a function
is even worse since the pickle cannot address it by name. If you create the
namedtuple type in a real module that is available to both interpreters, it will
> * See if you can come up with a super simple example that displays this error
> (isolate it).
> * See if you can figure out where the TypeError is coming from.
It is possible that there is a name collision.
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
More information about the IPython-User