[Numpy-discussion] Cython issues w/ 1.4.0
Tue Dec 8 15:25:50 CST 2009
ti, 2009-12-08 kello 21:52 +0100, Dag Sverre Seljebotn kirjoitti:
> How about this:
> - Cache/store the format string in a bytes object in a global
> WeakRefKeyDict (?), keyed by dtype
> - The array holds a ref to the dtype, and the Py_buffer holds a ref to
> the array (through the obj field).
Yep, storage in a static variable is the second alternative. We can even
handle allocation and deallocation manually.
I think I'll make it this way then.
> Alternatively, create a new Python object and stick it in the "obj" in
> the Py_buffer, I don't think obj has to point to the actual object the
> buffer was acquired from, as long as it keeps alive a reference to it
> somehow (though I didn't find any docs for the obj field, it was added
> as an afterthought by the implementors after the PEP...). But the only
> advantage is not using weak references (if that is a problem), and it is
> probably slower and doesn't cache the string.
The current implementation of MemoryView in Python assumes that you can
But this can be changed, IIRC, the memoryview has also a ->base member
containing the same info.
> >> - We need to cache the buffer protocol format string somewhere,
> >> if we do not want to regenerate it on each buffer acquisition.
> > My suspicion is that YAGNI. I would wait until it is actually in use
> > and we see whether it takes up a significant amount of time in actual
> > code.
Sure, it's likely that it won't be a real problem performance-wise, as
it's simple C code.
The point is that the format string needs to be stored somewhere for
later deallocation, and to work around bugs in Python, we cannot put it
in Py_buffer where it would naturally belong to.
But anyway, it may really be best to not pollute object structs because
of a need for workarounds -- I suppose if I submit patches to Python
soon, they may make it in releases before Numpy 1.5.0 rolls out. For
backward compatibility, we'll just make do with static variables.
Ok, the reserved-for-future pointers in structs may not then be needed
after all, at least for this purpose.
More information about the NumPy-Discussion