[Numpy-discussion] Question about improving genfromtxt errors
Fri Sep 25 18:30:41 CDT 2009
On Fri, Sep 25, 2009 at 3:51 PM, Christopher Barker
> One more thought:
> Pierre GM wrote:
>>>> That way, we don't slow
>>>> things down when everything works,
> how long can it really take to increment an integer as each line is
> parsed? I'd suspect no one would even notice!
A 1000 converters later...
FWIW, I have a script that creates and savez arrays from several text
files in total about 1.5 GB of text.
without the incrementing in genfromtxt
Run time: 122.043943 seconds
with the incrementing in genfromtxt
Run time: 131.698873 seconds
If we just want to always keep track of things, I would be willing to
take a poorly measured 8 % slowdown, because the info that I took from
the errors is the only thing that made what I was doing at all
More information about the NumPy-Discussion