[Numpy-discussion] How to avoid extra copying when forming an array from an iterator

srean srean.list@gmail....
Fri Jun 24 14:38:18 CDT 2011


A valiant exercise in hope:

Is this possible to do it without a loop or extra copying. What I have is an
iterator that yields a fixed with string on every call to next(). Now I want
to create a numpy array of ints out of the last 4 chars of that string.

My plan was to pass the iterator through a generator that returned an
iterator over the last 4 chars. (sub question: given that strings are
immutable, is it possible to yield a view of the last 4 chars rather than a
copy). Then apply StringIO.writelines() on the 2-char iterator returned.
After its done, create a numpy.array from the StringIO's buffer.

This does not work, the other option is to use an array.array in place of a
StringIO object. But is it possible to fill an array.array using a lazy
iterator without an explicit loop in python. Something like the writelines()
call

I know premature optimization and all that, but this indeed needs to be done
efficiently

Thanks again for your gracious help

-- srean

On Fri, Jun 24, 2011 at 9:12 AM, Robert Kern <robert.kern@gmail.com> wrote:

> On Fri, Jun 24, 2011 at 04:03, srean <srean.list@gmail.com> wrote:
> > To answer my own question, I guess I can keep appending to a
> array.array()
> > object and get a numpy.array from its buffer if possible.  Is that the
> > efficient way.
>
> It's one of the most efficient ways to do it, yes, especially for 1D
> arrays.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20110624/f66b1369/attachment.html 


More information about the NumPy-Discussion mailing list