[SciPy-User] Numpy pickle format
Wed Nov 24 16:20:05 CST 2010
On 11/24/10 2:00 PM, David Baddeley wrote:
> I was wondering if anyone could point me to any documentation for the (binary)
> format of pickled numpy arrays.
> To put my request into context, I'm using Pyro to communicate between python and
> jython, and would like push numpy arrays into the python end and pull something
> I can work with in jython out the other end
maybe the native (*.npy) format would be easier to deal with.
And you can pack a bunch of those together in a zip file with savez.
If Jython has a struct and array.array (or SOME sort of binary format
suitable for storing the data), it would be pretty easy to unpack them
(I was thinking of a minimal class
> wrapping the std libraries array.array, and having some form of shape property
> (I can pretty much guarantee that the data going in is c-contiguous, so there
> shouldn't be any strides nastiness).
> The proper way to do this would be to convert my numpy arrays to this minimal
> wrapper before pushing them onto the wire, but I've already got a fair bit of
> python code which pushes arrays round using Pyro, which I'd prefer not to have
> to rewrite. The pickle representation of array.array is also slightly different
> (broken) between cPython and Jython, and although you can pickle and unpickle,
> you end up swapping the endedness, so to recover the data [in the Jython ->
> Python direction] you've got to create a numpy array and then a view of that
> with reversed endedness.
> What I was hoping to do instead was to construct a dummy numpy.ndarray class in
> jython which knew how to pickle/unpickle numpy arrays.
> The ultimate goal is to create a Python -> ImageJ bridge so I can push images
> from some python image processing code I've got across into ImageJ without
> having to manually save and open the files.
> Would appreciate any suggestions,
> SciPy-User mailing list
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
More information about the SciPy-User