[Numpy-discussion] Github migration?

Jason McCampbell jmccampbell@enthought....
Tue Aug 31 15:56:04 CDT 2010

Hi Chuck (and anyone else interested),

I updated the refactoring page on the NumPy developer wiki (seems to be down
or I'd paste in the link).  It certainly isn't complete, but there are a lot
more details about the data structures and memory handling and an outline of
some additional topics that needs to be filled in.


On Wed, Aug 25, 2010 at 10:20 AM, Charles R Harris <
charlesr.harris@gmail.com> wrote:

> On Wed, Aug 25, 2010 at 8:19 AM, Jason McCampbell <
> jmccampbell@enthought.com> wrote:
>> Chuck,
>> I will update the wiki page on the Numpy developer site that discusses the
>> refactoring this week.  Right now what's there reflects our plans before
>> they met the reality of code.  Needless to say, the actual implementation
>> differs in some of the details.
>> Here is a very brief overview of the structure:
>> - The libndarray directory now contains all of the code for the 'core'
>> library.  This library is independent of Python and implements most of the
>> array, descriptor, iterator, and ufunc functionality.  The goal is that all
>> non-trivial behavior should be in here, but in reality some parts are tied
>> fairly tightly to the CPython interpreter and will take more work to move
>> into the core.
>> - numpy/core/src/multiarray and numpy/core/src/umath now implement "just"
>> the CPython interface to libndarray.  We have preserved both the Python
>> interface and the C API.  Ideally each C API function is just a simple
>> wrapper around a call to the core API, though it doesn't always work out
>> that way. However, a large amount of code has been moved out of these
>> modules into the core.
>> - The core is built as a shared library that is independent of any given
>> interface layer.  That is, the same shared library/DLL can be used with
>> CPython, IronPython and any other implementation.  Each interface is
>> required to pass in a set of callbacks for handling reference counting,
>> object manipulation, and other interface-specific behavior.
>> - The core implements its own reference counting semantics that happen to
>> look very much like CPython's.  This was necessary to make the core library
>> independent of the interface layer and preserve performance (ie, limit the
>> number of callbacks).  The handshaking between interface and core is a bit
>> complicated but ends up working nicely and efficiently for both reference
>> counted and garbage collected systems.  I'll write up the details on the
>> wiki page.
>> - As Travis mentioned we are also working on a .NET back end to Cython.
>>  This lets us port the modules such as MTRAND without having to have two
>> separate interfaces, a Cython and a .NET version.  Instead, we can modify
>> the existing .pyx file to use the new core API (should improve performance
>> in CPython version slightly).  Once done, Cython can generate the .NET and
>> CPython interfaces from the same .pyx file.
>> We have done a fair amount of cleanup on the naming conventions but
>> certainly more needs to be done!
>> I'll write it up for everyone this week but feel free to email me with
>> other questions.
> Thanks for the summary, it clarifies things a lot. On my cleanup wish list,
> some of the functions use macros that contain jumps, which is not so nice.
> I've been intending to scratch that itch for several years now but haven't
> gotten around to it. I expect such things have a lower priority than getting
> the basic separation of functionality in place, but just in case...
> <snip>
> Chuck
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20100831/da3d4a2b/attachment-0001.html 

More information about the NumPy-Discussion mailing list