[IPython-User] ipython parallelism returning user-defined types
Wed Nov 10 15:39:24 CST 2010
On Wed, Nov 10, 2010 at 11:20 AM, Robert Kern <email@example.com> wrote:
> On 11/10/10 11:33 AM, Brian Granger wrote:
> > John,
> > On Tue, Nov 9, 2010 at 2:18 AM, John Reid <firstname.lastname@example.org
> > <mailto:email@example.com>> wrote:
> > Hi,
> > I'm having some problems returning named tuples from a multi-engine
> > client. I do something like:
> > from IPython.kernel import client
> > mec = client.MultiEngineClient()
> > @mec.parallel()
> > def parallel_get_starts(arg):
> > <do some work>
> > return <some named tuples>
> > results = parallel_get_starts(<some list>)
> > but I always get errors like:
> > TypeError: __new__() takes exactly 8 arguments (2 given)
> > Hmm, that is an odd error that I have never seen. Even the fact that it
> is a
> > TypeError is odd to me. A couple of things to look at:
> > * Make sure that named tuples can be pickled and unpickled.
> namedtuple objects can be pickled, but you have to be particularly careful.
> Creating a namedtuple type at the prompt and then trying to send that over
> work because the type won't be available on the other side. IPython's
> will cause more complications. Creating a namedtuple type inside of a
> is even worse since the pickle cannot address it by name. If you create the
> namedtuple type in a real module that is available to both interpreters, it
> probably work.
Robert makes good points. You do have to make sure types are in modules
when using IPython. I would put namedtuple in a module, and then make sure
you import that module in both the client and engine namespaces.
> > * See if you can come up with a super simple example that displays this
> > (isolate it).
> > * See if you can figure out where the TypeError is coming from.
> It is possible that there is a name collision.
Yes, that sounds plausible. Putting namedtuple in a module will also help
any name collisions.
> Robert Kern
> "I have come to believe that the whole world is an enigma, a harmless
> that is made terrible by our own mad attempt to interpret it as though it
> an underlying truth."
> -- Umberto Eco
> IPython-User mailing list
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-User