[Numpy-discussion] Possible roadmap addendum: building better text file readers

Erin Sheldon erin.sheldon@gmail....
Thu Feb 23 14:55:15 CST 2012


Excerpts from Wes McKinney's message of Thu Feb 23 15:45:18 -0500 2012:
> Reasonably wide CSV files with hundreds of thousands to millions of
> rows. I have a separate interest in JSON handling but that is a
> different kind of problem, and probably just a matter of forking
> ultrajson and having it not produce Python-object-based data
> structures.

As a benchmark, recfile can read an uncached file with 350,000 lines and
32 columns in about 5 seconds.  File size ~220M

-e
-- 
Erin Scott Sheldon
Brookhaven National Laboratory


More information about the NumPy-Discussion mailing list