[Numpy-discussion] Github migration?

Jason McCampbell jmccampbell@enthought....
Thu Aug 26 09:33:33 CDT 2010


On Wed, Aug 25, 2010 at 10:20 AM, Charles R Harris <
charlesr.harris@gmail.com> wrote:

>
>
> On Wed, Aug 25, 2010 at 8:19 AM, Jason McCampbell <
> jmccampbell@enthought.com> wrote:
>
>> Chuck,
>>
>> I will update the wiki page on the Numpy developer site that discusses the
>> refactoring this week.  Right now what's there reflects our plans before
>> they met the reality of code.  Needless to say, the actual implementation
>> differs in some of the details.
>>
>> Here is a very brief overview of the structure:
>>
>> - The libndarray directory now contains all of the code for the 'core'
>> library.  This library is independent of Python and implements most of the
>> array, descriptor, iterator, and ufunc functionality.  The goal is that all
>> non-trivial behavior should be in here, but in reality some parts are tied
>> fairly tightly to the CPython interpreter and will take more work to move
>> into the core.
>>
>> - numpy/core/src/multiarray and numpy/core/src/umath now implement "just"
>> the CPython interface to libndarray.  We have preserved both the Python
>> interface and the C API.  Ideally each C API function is just a simple
>> wrapper around a call to the core API, though it doesn't always work out
>> that way. However, a large amount of code has been moved out of these
>> modules into the core.
>>
>> - The core is built as a shared library that is independent of any given
>> interface layer.  That is, the same shared library/DLL can be used with
>> CPython, IronPython and any other implementation.  Each interface is
>> required to pass in a set of callbacks for handling reference counting,
>> object manipulation, and other interface-specific behavior.
>>
>> - The core implements its own reference counting semantics that happen to
>> look very much like CPython's.  This was necessary to make the core library
>> independent of the interface layer and preserve performance (ie, limit the
>> number of callbacks).  The handshaking between interface and core is a bit
>> complicated but ends up working nicely and efficiently for both reference
>> counted and garbage collected systems.  I'll write up the details on the
>> wiki page.
>>
>> - As Travis mentioned we are also working on a .NET back end to Cython.
>>  This lets us port the modules such as MTRAND without having to have two
>> separate interfaces, a Cython and a .NET version.  Instead, we can modify
>> the existing .pyx file to use the new core API (should improve performance
>> in CPython version slightly).  Once done, Cython can generate the .NET and
>> CPython interfaces from the same .pyx file.
>>
>> We have done a fair amount of cleanup on the naming conventions but
>> certainly more needs to be done!
>>
>> I'll write it up for everyone this week but feel free to email me with
>> other questions.
>>
>>
> Thanks for the summary, it clarifies things a lot. On my cleanup wish list,
> some of the functions use macros that contain jumps, which is not so nice.
> I've been intending to scratch that itch for several years now but haven't
> gotten around to it. I expect such things have a lower priority than getting
> the basic separation of functionality in place, but just in case...
>

Yes, I know which ones you are talking about -- both goto's and returns.
 They have been bugging me, too.  A few uses have been fixed, but I would
like to clean the rest up.  We have also been trying to simplify some of the
naming to reduce duplication.


> How do you manage PyCapsule/PyCObject? I don't recall how deeply they were
> used but ISTR that they were used below the top level interface layer in
> several places.
>

Many or most of the uses of them have been removed.  There were several
instances where either a PyCapsule or a tuple with fixed content was used
and need to be accessed in the core. In this cases we just defined a new
struct with the appropriate fields. The capsule types never make it to the
core and I'd have to do a search to see where they are even used now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20100826/a64e22d5/attachment-0001.html 


More information about the NumPy-Discussion mailing list