[Numpy-discussion] Re: Bytes Object and Metadata
perry at stsci.edu
Mon Mar 28 13:54:10 CST 2005
On Mar 28, 2005, at 4:30 AM, Travis Oliphant wrote:
> Thank you for your detailed explanations. This is starting to make
> more sense to me. It is obvious that you understand what we are
> trying to do, and I pretty much agree with you in how you think it
> should be done. I think you do a great job of explaining things.
> I agree we should come up with a set of names for the interface to
> arrayobjects. I'm even convinced that offset should be an optional
> part of the interface (implied 0 if it's not there).
Just to add my two cents, I don't think I ever thought it was necessary
to bundle the metadata with the memory object for the reasons Scott
outlined. It isn't needed functionally, and there are cases where the
same memory may be used in different contexts (as is done with our
Numarray, when it uses the buffer object, always gets a fresh pointer
for the buffer object for every data access. But Scott is right that
that pointer is good so long as there isn't a chance for something else
to change it. In practice, I don't think that ever happens with the
buffers that numarray happens to use, but it's still a flaw of the
current buffer object that there is no way to ensure it won't change.
I'm not sure how the support for large data sets should be handled. I
generally think that it will be very awkward to handle these until
Python does as well. Speaking of which...
I had been in occasional contact with Martin von Loewis about his work
to update Python to handle 64-bit addressing. We weren't planning to
handle this in nummarray (nor Numeric3, right Travis or do I have that
wrong?) until Python did. A few months ago Martin said he was mostly
done. I had a chance to talk to him at Pycon about where that work
stood. Unfortunately, it is not turning out to be as easy as he hoped.
This is too bad. I have a feeling that this work is going to stall
without help on our (numpy community) part to help make the changes or
drum beating to make it a higher priority. At the moment the Numeric3
effort should be the most important focus, but I think that after that,
this should become a high priority.
More information about the Numpy-discussion