[Numpy-discussion] Github migration?
Thu Aug 26 09:33:33 CDT 2010
On Wed, Aug 25, 2010 at 10:20 AM, Charles R Harris <
> On Wed, Aug 25, 2010 at 8:19 AM, Jason McCampbell <
> email@example.com> wrote:
>> I will update the wiki page on the Numpy developer site that discusses the
>> refactoring this week. Right now what's there reflects our plans before
>> they met the reality of code. Needless to say, the actual implementation
>> differs in some of the details.
>> Here is a very brief overview of the structure:
>> - The libndarray directory now contains all of the code for the 'core'
>> library. This library is independent of Python and implements most of the
>> array, descriptor, iterator, and ufunc functionality. The goal is that all
>> non-trivial behavior should be in here, but in reality some parts are tied
>> fairly tightly to the CPython interpreter and will take more work to move
>> into the core.
>> - numpy/core/src/multiarray and numpy/core/src/umath now implement "just"
>> the CPython interface to libndarray. We have preserved both the Python
>> interface and the C API. Ideally each C API function is just a simple
>> wrapper around a call to the core API, though it doesn't always work out
>> that way. However, a large amount of code has been moved out of these
>> modules into the core.
>> - The core is built as a shared library that is independent of any given
>> interface layer. That is, the same shared library/DLL can be used with
>> CPython, IronPython and any other implementation. Each interface is
>> required to pass in a set of callbacks for handling reference counting,
>> object manipulation, and other interface-specific behavior.
>> - The core implements its own reference counting semantics that happen to
>> look very much like CPython's. This was necessary to make the core library
>> independent of the interface layer and preserve performance (ie, limit the
>> number of callbacks). The handshaking between interface and core is a bit
>> complicated but ends up working nicely and efficiently for both reference
>> counted and garbage collected systems. I'll write up the details on the
>> wiki page.
>> - As Travis mentioned we are also working on a .NET back end to Cython.
>> This lets us port the modules such as MTRAND without having to have two
>> separate interfaces, a Cython and a .NET version. Instead, we can modify
>> the existing .pyx file to use the new core API (should improve performance
>> in CPython version slightly). Once done, Cython can generate the .NET and
>> CPython interfaces from the same .pyx file.
>> We have done a fair amount of cleanup on the naming conventions but
>> certainly more needs to be done!
>> I'll write it up for everyone this week but feel free to email me with
>> other questions.
> Thanks for the summary, it clarifies things a lot. On my cleanup wish list,
> some of the functions use macros that contain jumps, which is not so nice.
> I've been intending to scratch that itch for several years now but haven't
> gotten around to it. I expect such things have a lower priority than getting
> the basic separation of functionality in place, but just in case...
Yes, I know which ones you are talking about -- both goto's and returns.
They have been bugging me, too. A few uses have been fixed, but I would
like to clean the rest up. We have also been trying to simplify some of the
naming to reduce duplication.
> How do you manage PyCapsule/PyCObject? I don't recall how deeply they were
> used but ISTR that they were used below the top level interface layer in
> several places.
Many or most of the uses of them have been removed. There were several
instances where either a PyCapsule or a tuple with fixed content was used
and need to be accessed in the core. In this cases we just defined a new
struct with the appropriate fields. The capsule types never make it to the
core and I'd have to do a search to see where they are even used now.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion