[Numpy-discussion] Help using numPy to create a very large multi dimensional array
Thu Apr 19 10:58:45 CDT 2007
Lisandro Dalcin wrote:
> I am also +1 on this, but this functionality should be implemented in
> C, I think.
> I've just tested numpy.fromfile('name.txt', sep=' ')
> against pylab.load('name.txt') for a 35MB text file, the number are:
> numpy.fromfile: 2.66 sec.
> pylab.load: 16.64 sec.
exactly that's expected. fromfile is designed to do the easy cases as
fast as possible, pylab.load is designed to be be flexible, I'm not user
you need both the speed and flexibility at the same time.
By the way, I haven't looked at pylab.load() for a while, but it could
perhaps be sped up by using fromfile() and or fromstring internally.
There may be some opportunity to special case the easy ones too (i.e.
all columns desired, etc.)
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
More information about the Numpy-discussion