[Numpy-discussion] advanced indexing bug with huge arrays?

David Warde-Farley wardefar@iro.umontreal...
Mon Jan 23 14:21:32 CST 2012


Hi Travis,

Thanks for your reply.

On Mon, Jan 23, 2012 at 01:33:42PM -0600, Travis Oliphant wrote:
> Can you determine where the problem is, precisely.    In other words, can you verify that c is not getting filled in correctly? 
> 
> You are no doubt going to get overflow in the summation as you have a uint8 parameter.   But, having that overflow be exactly '0' would be surprising.  

I've already looked at this actually. The last 4400000 or so rows of c are
all zero, however 'a' seems to be filled in fine:

>>> import numpy
>>> a = numpy.array(numpy.random.randint(256,size=(5000000,972)),
>>> dtype=numpy.uint8)
>>> b = numpy.random.randint(5000000,size=(4993210,))
>>> c = a[b]
>>> print c
[[186 215 204 ..., 170  98 198]
 [ 56  98 112 ...,  32 233   1]
 [ 44 133 171 ..., 163  35  51]
 ..., 
 [  0   0   0 ...,   0   0   0]
 [  0   0   0 ...,   0   0   0]
 [  0   0   0 ...,   0   0   0]]
>>> print a
[[ 30 182  56 ..., 133 162 173]
 [112 100  69 ...,   3 147  80]
 [124  70 232 ..., 114 177  11]
 ..., 
 [ 22  42  31 ..., 141 196 134]
 [ 74  47 167 ...,  38 193   9]
 [162 228 190 ..., 150  18   1]]

So it seems to have nothing to do with the sum, but rather the advanced
indexing operation. The zeros seem to start in the middle of row 574519,
in particular at element 356. This is reproducible with different random
vectors of indices, it seems.

So 558432824th element things go awry. I can't say it makes any sense to
me why this would be the magic number.

David


More information about the NumPy-Discussion mailing list