[SciPy-User] Numpy pickle format

Christopher Barker Chris.Barker@noaa....
Wed Nov 24 16:20:05 CST 2010


On 11/24/10 2:00 PM, David Baddeley wrote:
> I was wondering if anyone could point me to any documentation for the (binary)
> format of pickled numpy arrays.
>
> To put my request into context, I'm using Pyro to communicate between python and
> jython, and would like push numpy arrays into the python end and pull something
> I can work with in jython out the other end

maybe the native  (*.npy) format would be easier to deal with.

http://svn.scipy.org/svn/numpy/trunk/doc/neps/npy-format.txt

And you can pack a bunch of those together in a zip file with savez.

If Jython has a struct and array.array (or SOME sort of binary format 
suitable for storing the data), it would be pretty easy to unpack them 
in Jython.

-Chris



  (I was thinking of a minimal class
> wrapping the std libraries array.array, and having some form of shape property
> (I can pretty much guarantee that the data going in is c-contiguous, so there
> shouldn't be any strides nastiness).
>
> The proper way to do this would be to convert my numpy arrays to this minimal
> wrapper before pushing them onto the wire, but I've already got a fair bit of
> python code which pushes arrays round using Pyro, which I'd prefer not to have
> to rewrite. The pickle representation of array.array is also slightly different
> (broken) between cPython and Jython, and although you can pickle and unpickle,
> you end up swapping the endedness, so to recover the data [in the Jython ->
> Python direction] you've got to create a numpy array and then a view of that
> with reversed endedness.
>
> What I was hoping to do instead was to construct a dummy numpy.ndarray class in
> jython which knew how to pickle/unpickle  numpy arrays.
>
> The ultimate goal is to create a Python ->  ImageJ bridge so I can push images
> from some python image processing code I've got across into ImageJ without
> having to manually save and open the files.
>
> Would appreciate any suggestions,
>
> thanks,
> David
>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov


More information about the SciPy-User mailing list