[Numpy-discussion] Buffer interface PEP
Tue Mar 27 16:21:09 CDT 2007
I have a specific question and then a general question, and some
minor issues for clarification.
Specifically, regarding the arguments to getbufferproc:
> 166 format
> 167 address of a format-string (following extended struct
> 168 syntax) indicating what is in each element of
> 169 of memory. The number of elements is len / itemsize,
> 170 where itemsize is the number of bytes implied by the format.
> 171 NULL if not needed in which case format is "B" for
> 172 unsigned bytes. The memory for this string must not
> 173 be freed by the consumer --- it is managed by the exporter.
Is this saying that either NULL or a pointer to "B" can be supplied
by getbufferproc to indicate to the caller that the array is unsigned
bytes? If so, is there a specific reason to put the (minor)
complexity of handling this case in the caller's hands, instead of
dealing with it internally to getbufferproc? In either case, the
wording is a bit unclear, I think.
The general question is that there are several other instances where
getbufferproc is allowed to return ambiguous information which must
be handled on the client side. For example, C-contiguous data can be
indicated either by a NULL strides pointer or a pointer to a properly-
constructed strides array. Clients that can't handle C-contiguous
data (contrived example, I know there is a function to deal with
that) would then need to check both for NULL *and* inside the strides
array if not null, before properly deciding that the data isn't
usable them. Similarly, the suboffsets can be either all negative or
NULL to indicate the same thing.
Might it be more appropriate to specify only one canonical behavior
in these cases? Otherwise clients which don't do all the checks on
the data might not properly interoperate with providers which format
these values in the alternate manner.
Also, some typos, and places additional clarification could help:
> 253 PYBUF_STRIDES (strides and isptr)
Should 'isptr' be 'suboffsets'?
> 75 of a larger array can be described without copying the data. T
> 279 Get the buffer and optional information variables about the
> 280 Return an object-specific view object (which may be simply a
> 281 borrowed reference to the object itself).
This phrasing (and similar phrasing elsewhere) is somewhat opaque to
me. What's an "object-specific view object"?
> 287 Call this function to tell obj that you are done with your "view"
Similarly, the 'view' concept and terminology should be defined more
clearly in the preamble.
> 333 The struct string-syntax is missing some characters to fully
> 334 implement data-format descriptions already available elsewhere (in
> 335 ctypes and NumPy for example). Here are the proposed additions:
Is the following table just the additions? If so, it might be good to
show the full spec, and flag the specific additions. If not, then the
additions should be flagged.
> 341 't' bit (number before states how many bits)
> 372 According to the struct-module, a number can preceed a character
> 373 code to specify how many of that type there are. The
I'm confused -- could this be phrased more clearly? Does '5t' refer
to a field 5-bits wide, or 5-one bit fields? Is 'ttttt' allowed? If
so, is it equivalent to or different from '5t'?
> 378 Functions should be added to ctypes to create a ctypes object from
> 379 a struct description, and add long-double, and ucs-2 to ctypes.
In general, the logic of the 'locking mechanism' should be described
at a high level at some point. It's described in nitty-gritty
details, but at least I would have appreciated a bit more of a
discussion about the general how and why -- this would be helpful to
clients trying to use the locking mechanism properly.
Thanks to Travis and everyone else involved for your work on this
cogent and sorely-needed PEP.
On Mar 27, 2007, at 12:42 PM, Travis Oliphant wrote:
> Hi all,
> I'm having a good discussion with Carl Banks and Greg Ewing over on
> python-dev about the buffer interface. We are converging on a pretty
> good design, IMHO. If anybody wants to participate in the
> please read the PEP (there are a few things that still need to change)
> and add your two cents over on python-dev (you can read it through the
> gmane gateway and participate without signing up).
> The PEP is stored in numpy/doc/pep_buffer.txt on the SVN tree for
> Best regards,
> Numpy-discussion mailing list
More information about the Numpy-discussion