[Numpy-discussion] [Repost] Adding the arrays returned in an array iterator
Charles R Harris
Fri Jun 3 09:38:42 CDT 2011
On Thu, Jun 2, 2011 at 7:36 PM, srean <firstname.lastname@example.org> wrote:
> Bumping my question tentatively. I am fairly sure there is a good answer
> and for some reason it got overlooked.
> ---------- Forwarded message ----------
> From: srean <email@example.com>
> Date: Fri, May 27, 2011 at 10:36 AM
> Subject: Adding the arrays in an array iterator
> To: Discussion of Numerical Python <firstname.lastname@example.org>
> Hi List,
> I have to sum up an unknown number of ndarrays of the same size. These
> arrays, possibly thousands in number, are provided via an iterator. Right
> now I use python reduce with operator.add.
> Does that invoke the corresponding ufunc internally ? I want to avoid
> creating temporaries, which I suspect a naive invocation of reduce will
> create. With ufunc I know how to avoid making copies using the output
> parameter, but not sure how to make use of that in reduce.
> It is not essential that I use reduce though, so I would welcome idiomatic
> and efficient way of executing this. So far I have stayed away from building
> an ndarray object and summing across the relevant dimension. Is that what I
> should be doing ? Different invocations of this function has different
> number of arrays, so I cannot pre-compile away this into a numexpr.
There are several ways to do this depending on how big the arrays are, how
many there are, and how much memory you have. If you have all the arrays as
slices in a cube with the arrays numbered by the first index, then you can
If they are in a list, then I would do something like
result = arrays.copy()
for a in arrays[1:]:
result += a
But much depends on the details of your problem.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion