[Numpy-discussion] Aggregate memmap

Anne Archibald peridot.faceted@gmail....
Thu Apr 22 17:20:56 CDT 2010


On 21 April 2010 15:41, Matthew Turk <matthewturk@gmail.com> wrote:
> Hi there,
>
> I've quite a bit of unformatted fortran data that I'd like to use as
> input to a memmap, as sort of a staging area for selection of
> subregions to be loaded into RAM.  Unfortunately, what I'm running
> into is that the data was output as a set of "slices" through a 3D
> cube, instead of a single 3D cube -- the end result being that each
> slice also contains a record delimiter.  I was wondering if there's a
> way to either specify that every traversal through the least-frequent
> dimension requires an additional offset or to calculate individual
> offsets for each slice myself and then aggregate these into a "super
> memmap."

If you had a delimiter for each entry in a record array you could just
make a dummy record field for the delimiter, but it sounds like you
have one delimiter per slice instead. I think the way to go is to
memmap it in as a flat uint8 array and then construct a view into that
array with appropriate strides (which includes skipping the
delimiter). This may well be what you meant by a "super memmap".

> Thanks for any suggestions you might have!

Good luck,
Anne

> -Matt
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


More information about the NumPy-Discussion mailing list