[Numpy-discussion] Pattern for reading non-simple binary files

Robert Kern robert.kern@gmail....
Fri Jan 23 16:07:54 CST 2009


On Fri, Jan 23, 2009 at 15:31, Ryan May <rmay31@gmail.com> wrote:
> Hi,
>
> I'm trying to read in a data from a binary-formatted file. I have the data
> format, (available at:
> http://www1.ncdc.noaa.gov/pub/data/documentlibrary/tddoc/td7000.pdf if you're
> really curious), but it's not what I would consider simple, with a lot of
> different blocks and messages, some that are optional and some that have
> different formats depending on the data type.  My question is, has anyone dealt
> with data like this using numpy?

Yes!

> Have you found a good pattern for how to
> construct a numpy dtype dynamically to decode the different parts of the file
> appropriately as you go along?

I use mmap and create numpy arrays for each block using the ndarray
constructor with the appropriate offset parameter. There isn't much of
a pattern for constructing the dtypes except to use constructor
functions.

Good luck!

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


More information about the Numpy-discussion mailing list