[Numpy-discussion] How to avoid extra copying when forming an array from an iterator
Fri Jun 24 15:07:44 CDT 2011
On Fri, Jun 24, 2011 at 14:38, srean <email@example.com> wrote:
> A valiant exercise in hope:
> Is this possible to do it without a loop or extra copying. What I have is an
> iterator that yields a fixed with string on every call to next(). Now I want
> to create a numpy array of ints out of the last 4 chars of that string.
> My plan was to pass the iterator through a generator that returned an
> iterator over the last 4 chars. (sub question: given that strings are
> immutable, is it possible to yield a view of the last 4 chars rather than a
Yes, but there isn't much point to it.
> Then apply StringIO.writelines() on the 2-char iterator returned.
> After its done, create a numpy.array from the StringIO's buffer.
> This does not work, the other option is to use an array.array in place of a
> StringIO object. But is it possible to fill an array.array using a lazy
> iterator without an explicit loop in python. Something like the writelines()
Your generator that is yielding the last four bytes of the string
fragments is already a Python loop. Adding another is not much
additional overhead. But you can also be clever
a = array.array('c')
b = np.frombuffer(a, dtype=np.int32)
"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
-- Umberto Eco
More information about the NumPy-Discussion