[SciPy-User] IO of large ASCII table data
Alan G Isaac
Tue Aug 17 13:44:55 CDT 2010
> Dan Lussier wrote:
>> I am looking to read in large (many million rows) ASCII space
>> separated tables into numpy arrays.
On 8/17/2010 1:47 PM, Christopher Barker wrote:
> If it's all space separated, all one data type, and no comment lines or
> anything like that, then np.fromfile(file, sep=' ', dtype=np.float) is
> about as fast as you can get.
> fromfile() is very limited for reading text files, and doesn't handle
> error cases well, but if it works, it's fast.
Apparently TableIO has gotten harder to find, but:
Didn't check the posted license, but Mike Miller did
say he was open to a SciPy compatible license.
PS Sorry for the delayed reply...
More information about the SciPy-User