[Numpy-discussion] Pickling ufuncs?

James A. Bednar jbednar at inf.ed.ac.uk
Mon Jan 29 15:57:33 CST 2007


Hi,

Does anyone know whether it is possible to pickle and unpickle numpy
ufuncs?  I can't find anything about that on scipy.org or the mailing
list archives.  I have several important pieces of code that accept a
numpy ufunc as an argument and later apply it to some data, while
keeping a copy of the ufunc in an attribute.  So far I have been able
to pickle this code only by doing some very ugly hacks and
workarounds.

The ufuncs from Numeric 24 and earlier did not even deepcopy, which
caused us lots of other problems, but deepcopying them works now in
numpy 1.0.1. However, they still don't seem to pickle with the regular
picklers.  Is this deliberately disabled for some reason, or is there
some workaround?

Here's an example that illustrates the problem:


I have a class Test defined in test.py:

class Test(object):
    def __init__(self,a):
        self.a = a


If I try this code at the commandline:

import numpy
from test import Test
t=Test(numpy.multiply)
import pickle
s=pickle.dumps(t)

I get "TypeError: can't pickle ufunc objects"


Also, I can't reproduce it in a small example yet, but pickling
completes without errors in our larger program that also stores
ufuncs, yet I then get a different error on unpickling:

  File "/disk/home/lodestar1/jbednar/topographica/lib/python2.4/pickle.py", line 872, in load
    dispatch[key](self)
  File "/disk/home/lodestar1/jbednar/topographica/lib/python2.4/pickle.py", line 1097, in load_newobj
    obj = cls.__new__(cls, *args)
  TypeError: object.__new__(numpy.ufunc) is not safe, use numpy.ufunc.__new__()

Any ideas?  

Thanks,

Jim

_______________________________________________________________________________

Unix> cat test.py
class Test(object):
    def __init__(self,a):
        self.a = a
Unix> bin/python
Python 2.4.4 (#5, Nov  9 2006, 22:58:03) 
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> from test import Test
>>> t=Test(numpy.multiply)
>>> import pickle
>>> s=pickle.dumps(t)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/home/jb/lib/python2.4/pickle.py", line 1386, in dumps
    Pickler(file, protocol, bin).dump(obj)
  File "/home/jb/lib/python2.4/pickle.py", line 231, in dump
    self.save(obj)
  File "/home/jb/lib/python2.4/pickle.py", line 338, in save
    self.save_reduce(obj=obj, *rv)
  File "/home/jb/lib/python2.4/pickle.py", line 433, in save_reduce
    save(state)
  File "/home/jb/lib/python2.4/pickle.py", line 293, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/jb/lib/python2.4/pickle.py", line 663, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/home/jb/lib/python2.4/pickle.py", line 677, in _batch_setitems
    save(v)
  File "/home/jb/lib/python2.4/pickle.py", line 313, in save
    rv = reduce(self.proto)
  File "/home/jb/lib/python2.4/copy_reg.py", line 69, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle ufunc objects
>>> 


More information about the Numpy-discussion mailing list