[Numpy-discussion] numpy scalars and savez -- bug?
Chris Barker - NOAA Federal
Fri Apr 19 13:21:26 CDT 2013
On Fri, Apr 19, 2013 at 10:21 AM, Robert Kern <firstname.lastname@example.org> wrote:
> On Fri, Apr 19, 2013 at 8:45 PM, Chris Barker - NOAA Federal
> <email@example.com> wrote:
>> Given that numpy scalars do exist, and have their uses -- I found this
>> wiki page to remind me:
>> It would be nice if the .npy format could support them. Would that be
>> a major change? I'm trying to decide if this bugs me enough to work on
> I think that is significant scope creep for the .npy format, and I
> would like to avoid it.
hmm -- maybe it's more work that we want, but it seems to me that
numpy scalars are part and parcel of numpy -- so it makes sense for
.npy to save them.
> A case might be made for letting np.savez()
> simply pickle non-arrays that it is given. I have a vague recollection
> that that was discussed when savez() was designed and rejected as a
> moral hazard, but I could be wrong.
That could be a nice solution -- I'm not _so_ worried about moral hazards!
> The .npy and .npz formats are
> intentionally limited by design. As soon as you feel constrained by
> those limitations, you should start using more full-fledged and
> standard file formats.
well, maybe -- in this case, I'm using it to cache a bunch of data on
disk. The data are all in a dict of numpy arrays, so it was really
natural and easy (and I presume fast) to use npz. All I want to is
dump it to disk, and get back the same way. It worked great.
Then I needed a datetime stored with it all -- so I figured a
datetime64 scalar would be perfect. It's not a huge deal to use a
rank-zero array instead, but it would have been nicer to be able to
store a scalar (I suppose one trick may be that there are numpy
scalars, and there are regular old pyton scalars...)
Anyway -- going to HDF, or netcdf, or role-your-own really seems like
overkill for this. I just need something fast and simple and it
doesn't need to interchange with anything else.
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
More information about the NumPy-Discussion