[Numpy-discussion] fwrite() failure in PyArray_ToFile

Charles R Harris charlesr.harris@gmail....
Fri Sep 4 15:23:45 CDT 2009


On Fri, Sep 4, 2009 at 1:36 PM, David Warde-Farley <dwf@cs.toronto.edu>wrote:

> A friend of mine is trying to save approx 2GB of float32s with
> np.save, and it's been failing. I traced it to PyArray_ToFile in core/
> src/convert.c:
>
> Traceback (most recent call last):
>   File "preprocessTIMIT.py", line 302, in <module>
>     main()
>   File "preprocessTIMIT.py", line 299, in main
>     num.save("/ais/gobi1/gdahl/speech/%s" % k, d[k])
>   File "/nobackup/murray/bin/pylab-02/x86_64_GenuineIntel_6.06/
> Python-2.5.4/lib/python2.5/site-packages/numpy/lib/io.py", line 241,
> in save
>     format.write_array(fid, arr)
>   File "/nobackup/murray/bin/pylab-02/x86_64_GenuineIntel_6.06/
> Python-2.5.4/lib/python2.5/site-packages/numpy/lib/format.py", line
> 322, in write_array
>     array.tofile(fp)
> ValueError: 541110272 requested and 2028 written
>
> This is writing to a local ext3 filesystem. Apparently fwrite() is
> giving back values in the low thousands instead of the correct amount.
>
> Has anyone encountered this, or know why it would happen?       The values
> are 1004 and 2028 that we seem to keep seeing, which as far as I can
> tell are not error codes.
>
>
The odd values might be from the format code in the error message:

                PyErr_Format(PyExc_ValueError,
                        "%ld requested and %ld written",
                        (long) size, (long) n);

The code that is immediately responsible for the write is in lines 79-92 of
convert.c. You could do a bit of poking around in there to find out what is
happening.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20090904/b50e049b/attachment.html 


More information about the NumPy-Discussion mailing list