[Numpy-discussion] Load numarray with specific shape cause error

[Todd Miller]

On Fri, 2005-02-04 at 10:37 +0100, Vincent Picavet wrote:
> Hi,
> Here is what I launch in the Python interpreter :
> ----
> import numarray
> simulation_file = "/path/to/my/file.bin"
> a = numarray.fromfile(simulation_file, 'Float32',(3144, 33, 65))
> a.sum() 
> ----
> 
> And here I get the following error :
> 
> ----
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File
> "/nfs/data2/picavet/workspace/softs/python/lib/python2.4/site-packages/
> numarray/numarraycore.py", line 1133, in sum    return
> ufunc.add.reduce(ufunc.add.areduce(self, type=type).flat, type=type)
> libnumarray.error: add_Float64_reduce: access beyond buffer.
> offset=25151 buffersize=17160
> ----
> 
> Just after that, I execute :
> ----
> a.sum()
> 588108255.50844395
> a.mean()
> 87.206215933326803
> ----
> 
> ... and as you can see I got no error the second time.
> The binary file I try to load is a raw file of 3144*33*64
> 4-Bytes float elements.
> 
> In fact I just saw that if I use the shape (65,33,3144) when calling
> fromfile(), everything works just fine. And after having called a.sum()
> the first time, the shape of the array has been changed.
> 
> ----
> >>> a.getshape()
> (3144, 65, 33)
> >>> a.sum()
> Traceback (most recent call last):
> [snip, same as above]
> >>> a.getshape()
> (33, 65, 3144)
> ----
> 
> Any idea why I can't load an array with such a shape ?

Yes.  You're hitting a known bug which I'm working on now in preparation
for numarray-1.2.  

In a nutshell,  sum() exposes surprisingly complex code in numarray in
order to use higher precision without making a temporary copy of the
array.  This should be fixed when numarray-1.2 is released,  hopefully
very soon.

For now,  there are two work arounds:

# use simpler "same type" summing code at lower precision
a.sum('Float32')  

# create a higher precision temporary and sum that
a.astype('Float64').sum()  

Regards,
Todd