[Numpy-discussion] how to build a series of arrays as I go?
Tue Mar 18 13:05:00 CDT 2008
On Tue, Mar 18, 2008 at 4:25 AM, Chris Withers <email@example.com> wrote:
> Robert Kern wrote:
> > Appending to a list is almost always better than growing an array by
> > concatenation. If you have a real need for speed, though, there are a
> > few tricks you can do at the expense of complexity.
> I don't for this project but I might in future, where can I read about this?
There was a thread on one of the scipy lists several years ago, I
think. Before April 2005 certainly because I found a message from
myself referencing it.
Basically, if you are constructing a 1D array by appending individual
elements, the stdlib's array module is actually quite useful. It uses
the same preallocation strategy as lists. You then use
numpy.fromstring(buffer(pyarray), dtype=whatever) to create the numpy
If you are building up a 1D array by chunks instead of individual
elements, it probably depends on the type of the chunks. If the chunks
are already arrays, I believe that appending the chunks to a list and
using hstack() will be the best. If the chunks are still lists,
probably .extend()ing the accumulator list is probably best.
For (N>1)D arrays, append to lists.
"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
-- Umberto Eco
More information about the Numpy-discussion