[Numpy-discussion] Pickling ufuncs?

James A. Bednar jbednar at inf.ed.ac.uk
Mon Jan 29 15:57:33 CST 2007


Does anyone know whether it is possible to pickle and unpickle numpy
ufuncs?  I can't find anything about that on scipy.org or the mailing
list archives.  I have several important pieces of code that accept a
numpy ufunc as an argument and later apply it to some data, while
keeping a copy of the ufunc in an attribute.  So far I have been able
to pickle this code only by doing some very ugly hacks and

The ufuncs from Numeric 24 and earlier did not even deepcopy, which
caused us lots of other problems, but deepcopying them works now in
numpy 1.0.1. However, they still don't seem to pickle with the regular
picklers.  Is this deliberately disabled for some reason, or is there
some workaround?

Here's an example that illustrates the problem:

I have a class Test defined in test.py:

class Test(object):
    def __init__(self,a):
        self.a = a

If I try this code at the commandline:

import numpy
from test import Test
import pickle

I get "TypeError: can't pickle ufunc objects"

Also, I can't reproduce it in a small example yet, but pickling
completes without errors in our larger program that also stores
ufuncs, yet I then get a different error on unpickling:

  File "/disk/home/lodestar1/jbednar/topographica/lib/python2.4/pickle.py", line 872, in load
  File "/disk/home/lodestar1/jbednar/topographica/lib/python2.4/pickle.py", line 1097, in load_newobj
    obj = cls.__new__(cls, *args)
  TypeError: object.__new__(numpy.ufunc) is not safe, use numpy.ufunc.__new__()

Any ideas?  




Unix> cat test.py
class Test(object):
    def __init__(self,a):
        self.a = a
Unix> bin/python
Python 2.4.4 (#5, Nov  9 2006, 22:58:03) 
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> from test import Test
>>> t=Test(numpy.multiply)
>>> import pickle
>>> s=pickle.dumps(t)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/home/jb/lib/python2.4/pickle.py", line 1386, in dumps
    Pickler(file, protocol, bin).dump(obj)
  File "/home/jb/lib/python2.4/pickle.py", line 231, in dump
  File "/home/jb/lib/python2.4/pickle.py", line 338, in save
    self.save_reduce(obj=obj, *rv)
  File "/home/jb/lib/python2.4/pickle.py", line 433, in save_reduce
  File "/home/jb/lib/python2.4/pickle.py", line 293, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/jb/lib/python2.4/pickle.py", line 663, in save_dict
  File "/home/jb/lib/python2.4/pickle.py", line 677, in _batch_setitems
  File "/home/jb/lib/python2.4/pickle.py", line 313, in save
    rv = reduce(self.proto)
  File "/home/jb/lib/python2.4/copy_reg.py", line 69, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle ufunc objects

More information about the Numpy-discussion mailing list