[SciPy-user] Re: io.read_array slow

Christian Kristukat ckkart at hoc.net
Wed Sep 28 07:27:05 CDT 2005


Jordan H. Mantha wrote:
> Christian Kristukat wrote:
> 
> 
>>Hi,
>>I noticed that io.read_array is really slow compared to a python 
>>while/readline/split loop (about 5 times slower). I believe to remember 
>>that at some time it was written in C but when I looked at the source of 
>>scipy 0.3.2 it seems to be pure python. Is there a evident reason why it 
>>must be that slow or are there maybe plans to rewrite it in C?
>>Regards, 
>>Christian
> 
> 
> 
> I have noticed this too. I was wrote a fitting script that reads in a 3 
> column data file that has 502,000 rows. I first started with io.read_array 
> and it took > 10 min. to get the array in. I was reading "Python Scripting 
> for Computational Science" by Hans Petter Langtangen and used the 
> following:
> 
> 	data = array([float(x) for x in infile.read().split()], Float)
>         data.shape = (len(data)/3,3) 
> 

Nice short solution indeed. I think read_array could be cut down to something 
less versatile such that it can handle different separators and works at C speed.

Regards, Christian




More information about the SciPy-user mailing list