[SciPy-Dev] possible speed-up for arffread
Thu Jun 17 05:51:29 CDT 2010
On Wed, Jun 16, 2010 at 10:46 AM, Benjamin Root <email@example.com> wrote:
> I was looking at the scipy.io.arff module to see if I could easily shave
> some processing time for loading an ARFF file. Doing some profiling on a
> file with 40,000 floating point numbers pointed me to the safe_float()
> function in the arffread.py file. In it, it was stripping the string token
> of any whitespace and then comparing it to '?' (which is ARFF's missing data
> indicator). I found that if one just does a check for the '?' character,
> you can shave almost 30% of the processing time off of the safe_float()
> In addition, I found a very slight improvement by calculating the range(ni)
> once and reusing that variable in the generator function. Attached is my
> patch file.
> It isn't much, but it is noticeable.
> That looks good, thanks. I included it in
http://github.com/rgommers/scipy/tree/io, will commit that soon.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the SciPy-Dev