[SciPy-user] Python loops too slow
Ross Williamson
Ross.Williamson@usap....
Wed Apr 8 22:02:10 CDT 2009
Ok so this is really annoying and slowing my code down.
I've tried:
nsamp = zeros(711)
mask = ones(1000*1000)
binned_all is (1000*1000 set of indices including numbers between 0 and 710)
rbl = binned_ell.ravel()
rmask = mask.ravel()
nsamp[rbl] = nsamp[rbl] + mask
And all it does is just add 1 to each element of nsamp - i.e. nsamp is
now all ones rather than zeros and not the cumulative sum of the number
of elements in binned_all (Note I'm using one here to make life easy -
in reality it will be a float of different values)
Any help much appriciated
Ross
> Actually I've got another puzzler:
>
> I currently do the following to get an integrated vector from a 2d array:
>
> for i in xrange(ngrid):
> for j in xrange(ngrid):
> itemp = binned_ell[j,i]
> nsamp[itemp] = nsamp[itemp] + mask[j,i]
>
> where binned_ell is a set of indices and mask is just some numbers - Is
> there a more efficient way of doing this? I'm trying to work it out at
> the moment.
>
> Cheers
>
> Ross
>
>> Hi Guys
>>
>> Thanks for all your help - My code is now super quick and I'll implement
>> meshgrid in other areas where I've got double loops.
>>
>> Cheers
>>
>> Ross
>>
>>
>>> Pauli Virtanen <pav <at> iki.fi> writes:
>>>
>>>
>>>
>>>> i = arange(ngrid/2 + 1)
>>>> j = arange(ngrid/2 + 1)
>>>> result = 2*pi*hypot(i[None,:], j[:,None])/reso/ngrid
>>>>
>>>>
>>>>
>>>>
>>> It seems Pauli's solution may need to be repeated in the final output as below.
>>> Is that what you really want though because the submatrix isn't reflected
>>> uniformly - i.e. the first row and column are missing.
>>>
>>> Anyway, timings are below - it seems using meshgrid is more efficient than
>>> hypot.
>>>
>>> -Dave
>>>
>>> Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)]
>>> Type "copyright", "credits" or "license" for more information.
>>>
>>> IPython 0.9.1 -- An enhanced Interactive Python.
>>>
>>> Welcome to pylab, a matplotlib-based Python environment.
>>> For more information, type 'help(pylab)'.
>>>
>>> In [2]: def f1(ngrid, reso):
>>> ...: result = zeros([ngrid, ngrid])
>>> ...: for i in arange((ngrid / 2)+1):
>>> ...: for j in arange((ngrid / 2)+1):
>>> ...: result[j,i] = 2. * pi * sqrt(i ** 2. + j ** 2) / reso /
>>> (ngrid*1.0)
>>> ...: for i in xrange(ngrid):
>>> ...: result[i,ngrid / 2+1:] = result[i,1:(ngrid / 2)][::-1]
>>> ...: for i in xrange(ngrid):
>>> ...: result[ngrid / 2+1:,i] = result[1:(ngrid / 2),i][::-1]
>>> ...: return result
>>> ...: #
>>> ...:
>>>
>>> In [3]: def f2(ngrid, reso):
>>> ...: nsub = ngrid/2+1
>>> ...: i = np.arange(nsub)
>>> ...: j = np.arange(nsub)
>>> ...: submatrix = 2*pi*np.hypot(i[None,:], j[:,None])/reso/np.float(ngrid)
>>> ...: result = zeros([ngrid, ngrid])
>>> ...: result[0:nsub,0:nsub] = submatrix
>>> ...: result[0:nsub,nsub:] = np.fliplr(submatrix[:,1:-1])
>>> ...: result[nsub:,:] = np.flipud(result[1:nsub-1,:])
>>> ...: return result
>>> ...: #
>>> ...:
>>>
>>> In [4]: def f3(ngrid, reso):
>>> ...: nsub = ngrid/2+1
>>> ...: i, j = meshgrid(np.arange(nsub),np.arange(nsub))
>>> ...: submatrix = 2*pi*np.sqrt(i**2 + j**2)/reso/np.float(ngrid)
>>> ...: result = zeros([ngrid, ngrid])
>>> ...: result[0:nsub,0:nsub] = submatrix
>>> ...: result[0:nsub,nsub:] = np.fliplr(submatrix[:,1:-1])
>>> ...: result[nsub:,:] = np.flipud(result[1:nsub-1,:])
>>> ...: return result
>>> ...: #
>>> ...:
>>>
>>> In [5]: np.allclose(f1(100,1),f2(100,1))
>>> Out[5]: True
>>>
>>> In [6]: np.allclose(f1(100,1),f3(100,1))
>>> Out[6]: True
>>>
>>> In [7]:
>>>
>>> In [8]: timeit f1(100,1)
>>> 10 loops, best of 3: 60.6 ms per loop
>>>
>>> In [9]:
>>>
>>> In [10]: timeit f2(100,1)
>>> 1000 loops, best of 3: 840 µs per loop
>>>
>>> In [11]:
>>>
>>> In [12]: timeit f3(100,1)
>>> 1000 loops, best of 3: 258 µs per loop
>>>
>>> In [13]:
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> SciPy-user mailing list
>>> SciPy-user@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>>
>>>
>>>
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user@scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
More information about the SciPy-user
mailing list