[Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines

Bruce Southey bsouthey@gmail....
Wed Nov 4 17:56:03 CST 2009


On Wed, Nov 4, 2009 at 8:06 AM, Reckoner <reckoner@gmail.com> wrote:
> Here's an example:
>
> On winxp 64-bit:
>
> Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import numpy
>>>> import cPickle
>>>> a = numpy.eye(10)
>>>> cPickle.dump(a,open('from32bitxp.pkl','w'))
>>>> import numpy.core.multiarray
>>>> numpy.__version__
> '1.0.4'
>>>>
>
> On linux 64 bit:
>
> Python 2.5.4 (r254:67916, Feb  5 2009, 19:52:35)
> [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import numpy
>>>> import cPickle
>>>> cPickle.load(open('from32bitxp.pkl'))
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> ImportError: No module named multiarray
>>>> numpy.__version__
> '1.2.1'
>>>> import numpy.core.multiarray
>>>>
>
> Note that I transfer the from32bitxp file from the winxp32 machine to
> the linux host. Also, I've tried this with version 1.3 on winxp and
> get the same problem on the linux host.
>
> Here's more interesting info:
>
> On linux:
>
>>>> a = numpy.eye(10)
>>>> cPickle.dump(a,open('from64bitLinux.pkl','w'))
>
> upon transferring the file to winxp 32 and on winxp32:
>
>>>> cPickle.load(open('from64bitLinux.pkl'))
>
> See? No problem going from linux to winxp32; but problems going the other way.
>
> Please let me know if you need more info on this.
>
> Any help appreciated.
>
> On Tue, Nov 3, 2009 at 4:55 AM, Bruce Southey <bsouthey@gmail.com> wrote:
>> On Mon, Nov 2, 2009 at 6:31 PM, Reckoner <reckoner@gmail.com> wrote:
>>> thanks for the suggestion! I will look into it. The other thing is
>>> that the numpy arrays in question are actually embedded in another
>>> object. When I convert the numpy arrays into plain lists, and then
>>> cPickle them, there is no problem with any of the larger objects. That
>>> is the way we are currently working around this issue.
>>>
>>> Thanks again.
>>>
>>> On Mon, Nov 2, 2009 at 2:43 PM, Bruce Southey <bsouthey@gmail.com> wrote:
>>>> On Mon, Nov 2, 2009 at 2:42 PM, Reckoner <reckoner@gmail.com> wrote:
>>>>> Anybody have any ideas here?
>>>>>
>>>>> Otherwise, I'm thinking this should be posted to the numpy bugs list.
>>>>> What's the best way to report a bug of this kind?
>>>>>
>>>>> Thanks!
>>>>>
>>>>> On Fri, Oct 30, 2009 at 5:48 PM, Reckoner <reckoner@gmail.com> wrote:
>>>>>>> Robert Kern wrote:
>>>>>>> You can import numpy.core.multiarray on both machines?
>>>>>>
>>>>>> Yes. For each machine separately, you can cPickle files with numpy
>>>>>> arrays without problems loading/dumping. The problem comes from
>>>>>> transferring the win32 cPickle'd files to Linux 64 bit and then trying
>>>>>> to load them. Transferring cPickle'd files that do *not* have numpy
>>>>>> arrays work as expected. In other words, cPICKLE'd lists transfer fine
>>>>>> back and forth between the two machines. In fact, we currently get
>>>>>> around this problem by converting the numpy arrays to lists,
>>>>>> transferring them, and then re-numpy-ing them on the respective hosts
>>>>>>
>>>>>> thanks.
>>>>>>
>>>>>>
>>>>>> On Fri, Oct 30, 2009 at 11:13 AM, Reckoner <reckoner@gmail.com> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> % python -c 'import numpy.core.multiarray'
>>>>>>>
>>>>>>> works just fine, but when I try to load a file that I have transferred
>>>>>>> from another machine running Windows to one running Linux, I get:
>>>>>>>
>>>>>>> %  python -c 'import cPickle;a=cPickle.load(open("matrices.pkl"))'
>>>>>>>
>>>>>>> Traceback (most recent call last):
>>>>>>>  File "<string>", line 1, in <module>
>>>>>>> ImportError: No module named multiarray
>>>>>>>
>>>>>>> otherwise, cPickle works normally when transferring files that *do*
>>>>>>> not contain numpy arrays.
>>>>>>>
>>>>>>> I am using version 1.2 on both machines. It's not so easy for me to
>>>>>>> change versions, by the way, since this is the version that my working
>>>>>>> group has decided on to standardize on for this effort.
>>>>>>>
>>>>>>>
>>>>>>> Any help appreciated.
>>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion@scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>>>
>>>>
>>>> Have you have tried the other Cookbook approaches:
>>>> http://www.scipy.org/Cookbook/InputOutput
>>>> Like using numpy's own array io functions - load/save(z)?
>>>> (seems to work between 64-bit Windows 7 and 64-bit Linux - each has
>>>> different numpy versions)
>>>>
>>>> Bruce
>>>> _______________________________________________
>>
>> Can you provide you provide a small self-contained example of the
>> problem including object creation especially as your example does not
>> import numpy?
>>
>> Really you have to start at the beginning (like pickling and
>> transferring numpy arrays) and then increase the complexity to include
>> the object.
>>
>>
>> Bruce
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

Hi,
I did not see the file 'from32bitxp.pkl'. It would really help if you
can provide the full example that includes the creation of the
complete object so at least people could try doing the same process
with the object that you are using.

Bruce


More information about the NumPy-Discussion mailing list