[IPython-dev] First Performance Result

MinRK benjaminrk@gmail....
Thu Jul 22 15:59:42 CDT 2010


It would appear to be just the C implementation. Regular pickle takes
approximately the same amount of time as JSON.  The integer key issue is a
serious one.

Any JSON serialized dict with integer keys will get reconstructed
incorrectly with string keys. Approximately 100% of controller messages
include such a thing (keyed by engine IDs). I can get around it easily
enough in the controller/client code (in ugly ways), but it's user code that
I'm worried about. It means that we cannot, in general, allow user dicts to
be sent if we use JSON, unless on every send we walk all iterables and
convert every dict we find to a custom dict subclass, which is certainly
unacceptable.

This is not a problem with Python's JSON, it's a problem with JSON itself.

On Thu, Jul 22, 2010 at 13:31, Fernando Perez <fperez.net@gmail.com> wrote:

> Hey Min,
>
> On Thu, Jul 22, 2010 at 2:22 AM, MinRK <benjaminrk@gmail.com> wrote:
> >
> > It would appear that json is contributing 50% to the overall run time.
>
> any chance you could re-test using pickle instead of cPickle?  I want
> to see if the difference vs json is just from the faster C
> implementationo of cPickle.  If that's the case, we could later
> consider implementing a cython-based version of the json dump/load
> code.
>
> Cheers,
>
> f
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-dev/attachments/20100722/41985c87/attachment.html 


More information about the IPython-dev mailing list