[SciPy-User] Scipy views and slicing: Can I get a view-slice from only certain elements of an array?
Francesc Alted
faltet@pytables....
Tue Nov 2 05:32:52 CDT 2010
A Tuesday 02 November 2010 10:04:12 Jacob Biesinger escrigué:
> > Why not using a numpy.array object instead of array.array?
> > Indexing
> >
> > with them is much faster than using plain lists:
> >>>> a = np.arange(1000)
> >>>> b = np.arange(1e8)
> >>>> timeit b[a]
> >
> > 100000 loops, best of 3: 10.4 µs per loop
> >
> >>>> l = a.tolist()
> >>>> timeit b[l]
> >
> > 10000 loops, best of 3: 66.5 µs per loop
>
> Thanks for pointing this out-- using scipy.array instead of lists of
> ints shaved my iteration time from 4 minutes to 2.5 minutes.
>
> > NumPy arrays helps saving space too:
> >>>> sys.getsizeof(l)
> >
> > 8072
> >
> >>>> a.size*a.itemsize
> >
> > 8000 # 72 bytes less, not a lot but better than nothing
>
> My data actually looks more like lots of smallish lists. It looks
> like scipy.array and int lists take up about the same space.
> a = [range(10) for i in xrange(1000000)] # V=223m RES=182m
> a = [scipy.array(range(10), dtype=scipy.int32) for i in
> xrange(1000000)] # V=275m RES=192m
Yes. For arrays that small probably the overhead of numpy headers make
them to take more space than plain lists. The crosspoint for which
numpy containers consumes less memory starts is with 100's:
> a = [range(100) for i in xrange(100000)]
Memory usage: ******* list *******
VmSize: 170764 kB VmRSS: 98832 kB
VmData: 96304 kB VmStk: 180 kB
VmExe: 1352 kB VmLib: 10584 kB
> a = [np.array(range(100), dtype=np.int32) for i in xrange(100000)]
Memory usage: ******* numpy *******
VmSize: 136748 kB VmRSS: 64964 kB
VmData: 62292 kB VmStk: 176 kB
VmExe: 1352 kB VmLib: 10584 kB
--
Francesc Alted
More information about the SciPy-User
mailing list