[Numpy-discussion] Is there anyway to read raw binary file via pytable?
Wed Jul 28 15:09:51 CDT 2010
Francesc Alted wrote:
> A Wednesday 28 July 2010 18:05:11 脑关(BrainGateway)生命科学仪器 escrigué:
>> I had some 500GB numerical arrays
>> to process. Pytable claims to have some advance feature to enhance
>> processing speed and largely reduce physical memory requirement. However, I
>> do not wanna touch the raw data I had. Simply because I do not have doubled
>> diskspace to covert all 10TB data into HDF5. Is there any way to let
>> pytable read raw binary files or alternatively to package raw files into
>> HDF5 format without change the files themselves.?
> No, PyTables does not support raw data directly. You have to convert to HDF5
Which is probably worth it.
But if your data files are pretty simple, you may be able to use a
memory mapped array, or baring that, write your own reader with fromfile().
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
More information about the NumPy-Discussion