[Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number)
Tim Hochberg
tim.hochberg at ieee.org
Tue Oct 3 11:24:10 CDT 2006
David Cournapeau wrote:
> Hi,
>
> I was wondering if there was any way to speed up the following code:
>
> y = N.zeros((n, K))
> for i in range(K):
> y[:, i] = gauss_den(data, mu[i, :], va[i, :])
>
> Where K is of order 1e1, n of order 1e5. Normally, gauss_den is a quite
> expensive function, but the profiler tells me that the indexing y[:,i]
> takes almost as much time as the gauss_den computation (which computes n
> exp !). To see if the profiler is "right", i replaces with the (non
> valid) following function:
>
> y = N.zeros((n, K))
> for i in range(K):
> yt = gauss_den(data, mu[i, :], va[i, :])
> return y
>
> Where more than 99% of the code is spent inside gauss_den.
>
> I guess the problem is coming from the fact that y being C order, y[:,
> i] needs accessing data in a non 'linear' way. Is there a way to speed
> this up ? I did something like this:
>
> y = N.zeros((K, n))
> for i in range(K):
> y[i] = gauss_den(data, mu[i, :], va[i, :])
> return y.T
>
> which works, but I don't like it very much.
Why not?
> Isn't there any other way
That depends on the details of gauss_den.
A general note: for this kind of microoptimization puzzle, it's much
easier to help if you can post a self contained example, preferably
something fairly simple that still illustrates the speed issue, that we
can experiment with.
-tim
More information about the Numpy-discussion
mailing list