[Numpy-discussion] numpy.load raising IOError but EOFError expected
Ruben Salvador
rsalvador.wk@gmail....
Thu Jul 1 06:05:41 CDT 2010
Great! Thanks for all your answers!
I actually have the files created as .npy (appending a new array eact time).
I know it's weird, and it's not its intended use. But, for whatsoever
reasons, I came to use that. No turn back now.
Fortunately, I am able to read the files correctly, so being weird also, at
least, it works. Repeating the tests would be very time consuming. I'll just
try the different options mentioned for the following tests.
Anyway, I think this is a quite common situation. Tests running for a
loooooong time, producing results at very different times (not necessarily
huge amounts of data of results, it could be just a single float, or array),
and repeating these tests a lot of times, makes it absolutely necessary to
have numpyish functions/filetype to APPEND these freshly-new produced data
each time it is available. Having to load a .npz file, adding the new data
and saving again is wasting unnecesary resources. Having a single file for
each run of the test, though possible, for me, complicates the
post-processing section, while increasing the time to copy these files (many
small files tend to take longer to copy than one single bigger file). Why
not just a modified .npy filetype/function with a header indicating it's
hosting more than one array¿?
Cheers!
On Tue, Jun 29, 2010 at 12:43 AM, Friedrich Romstedt <
friedrichromstedt@gmail.com> wrote:
> 2010/6/28 Keith Goodman <kwgoodman@gmail.com>:
> > How about using h5py? It's not part of numpy but it gives you a
> > dictionary-like interface to your archive:
>
> Yeaa, or PyTables (is that equivalent)? It's also a hdf (or whatever,
> I don't recall precisely) interface.
>
> There were [ANN]s on the list about PyTables.
>
> Friedrich
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
--
Rubén Salvador
PhD student @ Centro de Electrónica Industrial (CEI)
http://www.cei.upm.es
Blog: http://aesatcei.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20100701/07a801bf/attachment.html
More information about the NumPy-Discussion
mailing list