[Numpy-tickets] [NumPy] #931: Records containing zero-length items pickle just fine, but cannot be unpickled
NumPy
numpy-tickets@scipy....
Tue Oct 14 00:22:25 CDT 2008
#931: Records containing zero-length items pickle just fine, but cannot be
unpickled
--------------------+-------------------------------------------------------
Reporter: njs | Owner: somebody
Type: defect | Status: new
Priority: normal | Milestone:
Component: Other | Version: 1.0.4
Severity: normal | Keywords:
--------------------+-------------------------------------------------------
In numpy 1.0.4 (I don't have any newer version convenient to test), it is
possible to create a structured array with a zero-repeat-count entry.
Such an array works fine, and can be pickled as normal. However,
attempting to unpickle such an array fails.
This is admittedly a somewhat esoteric bit of functionality, but the bug
shows up in a nasty way -- I had some data that took all night to generate
and that I saved to disk in a pickle, and it was only by chance that I
still had the original array in memory when I attempted to re-load the
data and discovered the problem. (And it contained a zero-length sub-
record because the code in question handles a variable number of
"distractor" variables, which happened to be 0 in this run.)
Single-line reproducer:
$ python -c 'import numpy, cPickle;
cPickle.loads(cPickle.dumps(numpy.array([(1, [])], dtype=[("a", "i"),
("b", "i", 0)])))'
Transcript:
{{{
~$ python
Python 2.5.2 (r252:60911, Jul 31 2008, 17:31:22)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.__version__
'1.0.4'
>>> a = np.array([(1, [])], dtype=[("a", np.int32), ("b", np.int32, 0)])
>>> a
array([(1, [])],
dtype=[('a', '<i4'), ('b', '<i4', 0)])
>>> import cPickle
>>> cPickle.dumps(a)
### (long string elided)
>>> cPickle.loads(cPickle.dumps(a))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: ('data type not understood', <type 'numpy.dtype'>, ('V0', 0,
1))
}}}
Thanks.
--
Ticket URL: <http://scipy.org/scipy/numpy/ticket/931>
NumPy <http://projects.scipy.org/scipy/numpy>
The fundamental package needed for scientific computing with Python.
More information about the Numpy-tickets
mailing list