[Numpy-discussion] advanced indexing bug with huge arrays?
David Warde-Farley
wardefar@iro.umontreal...
Mon Jan 23 14:21:32 CST 2012
Hi Travis,
Thanks for your reply.
On Mon, Jan 23, 2012 at 01:33:42PM -0600, Travis Oliphant wrote:
> Can you determine where the problem is, precisely. In other words, can you verify that c is not getting filled in correctly?
>
> You are no doubt going to get overflow in the summation as you have a uint8 parameter. But, having that overflow be exactly '0' would be surprising.
I've already looked at this actually. The last 4400000 or so rows of c are
all zero, however 'a' seems to be filled in fine:
>>> import numpy
>>> a = numpy.array(numpy.random.randint(256,size=(5000000,972)),
>>> dtype=numpy.uint8)
>>> b = numpy.random.randint(5000000,size=(4993210,))
>>> c = a[b]
>>> print c
[[186 215 204 ..., 170 98 198]
[ 56 98 112 ..., 32 233 1]
[ 44 133 171 ..., 163 35 51]
...,
[ 0 0 0 ..., 0 0 0]
[ 0 0 0 ..., 0 0 0]
[ 0 0 0 ..., 0 0 0]]
>>> print a
[[ 30 182 56 ..., 133 162 173]
[112 100 69 ..., 3 147 80]
[124 70 232 ..., 114 177 11]
...,
[ 22 42 31 ..., 141 196 134]
[ 74 47 167 ..., 38 193 9]
[162 228 190 ..., 150 18 1]]
So it seems to have nothing to do with the sum, but rather the advanced
indexing operation. The zeros seem to start in the middle of row 574519,
in particular at element 356. This is reproducible with different random
vectors of indices, it seems.
So 558432824th element things go awry. I can't say it makes any sense to
me why this would be the magic number.
David
More information about the NumPy-Discussion
mailing list