[Numpy-discussion] dealing with large arrays
Ivan Vilata i Balaguer
ivilata at carabos.com
Wed Jun 14 04:42:31 CDT 2006
En/na Mathew Yeates ha escrit::
> I typically deal with very large arrays that don't fit in memory. How
> does Numpy handle this? In Matlab I can use memory mapping but I would
> prefer caching as is done in The Gimp.
Hi Mathew. If you are in the need of storing large arrays on disk, you
may have a look at Pytables_. It will save you some headaches with the
on-disk representation of your arrays (it uses the self-describing HDF5
format), it allows you to load specific slices of arrays, and it
provides caching of data. The latest versions also support numpy.
Hope that helps,
.. _PyTables: http://www.pytables.org/
Ivan Vilata i Balaguer >qo< http://www.carabos.com/
Cárabos Coop. V. V V Enjoy Data
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 307 bytes
Desc: OpenPGP digital signature
Url : http://projects.scipy.org/pipermail/numpy-discussion/attachments/20060614/a39ff30e/attachment-0001.bin
More information about the Numpy-discussion