[SciPy-User] IO of large ASCII table data

Alan G Isaac alan.isaac@gmail....
Tue Aug 17 13:44:55 CDT 2010


> Dan Lussier wrote:
>> I am looking to read in large (many million rows) ASCII space
>> separated tables into numpy arrays.
  
On 8/17/2010 1:47 PM, Christopher Barker wrote:
> If it's all space separated, all one data type, and no comment lines or
> anything like that, then np.fromfile(file, sep=' ', dtype=np.float) is
> about as fast as you can get.
>
> fromfile() is very limited for reading text files, and doesn't handle
> error cases well, but if it works, it's fast.


Apparently TableIO has gotten harder to find, but:
http://kochanski.org/gpk/misc/TableIO.html

Didn't check the posted license, but Mike Miller did
say he was open to a SciPy compatible license.

Alan Isaac

PS Sorry for the delayed reply...


More information about the SciPy-User mailing list