[SciPy-user] future-safe saving of numpy arrays?
oliphant at ee.byu.edu
Thu Jun 8 14:50:21 CDT 2006
Cory Davis wrote:
> Hi All,
> I have had some trouble with my data and changes to numpy over time.
> I often want to save both single arrays and complicated objects with
> arrays as data members. Until now I have almost always used cPickle.
> But this can cause problems when I upgrade numpy/scipy, when I can no
> longer unpickle data saved using older versions.
Unfortunately, there were some bugs in the NumPy reduce implementation
that required small changes. Post 1.0, there will not be significant
changes made to pickle that cause old pickles not to load (I don't seen
any changes happening from now on, frankly). This is definitely a
growing pain of the pre-1.0 release.
> Does anyone have any suggestions on avoiding this problem?
A problem with Pickle generally, is that if you pickle objects requiring
specific modules, then any name changes in those modules will cause
difficulties with loading (most of these problems can be worked around
--- often trivially), but it does get to be a pain for long-term
persistence with Pickle. Using PyTables is probably a better idea.
More information about the SciPy-user