[Numpy-discussion] Fixing numpy 1.4.0 ABI breakage, and a plea for self-contained, small commits

David Cournapeau david@silveregg.co...
Wed Jan 27 18:57:37 CST 2010


Bruce Southey wrote:
> On Tue, Jan 26, 2010 at 11:02 PM, David Cournapeau
> <david@silveregg.co.jp> wrote:
>> Charles R Harris wrote:
>>
>>> Whatever we do, it would be good to figure out some way to avoid this
>>> problem in the future. We could hide access to the array, for instance.
>>> But again, that would require a lot of other code mods. Hmm...
>> That's something that we have to do at some point if we care about ABI
>> (I think we should care - expecting people to recompile all the
>> extensions for a new version of numpy is a big hindrance).
>>
>> Assuming python 1.5 will have py3k support, I was wondering about
>> starting working on NumPy 2.0, with massive changes to the C API so that
>> we can avoid this problem in the future: no more "naked" structures,
>> much cleaner/leaner headers to avoid accidental reliance on specific
>> private binary layouts, etc...
>>
>> David
> 
> Numpy 1.5? :-)
> 
> That was an incredible effort!
> 
> My understanding is that a minor numpy release should not break the
> ABI and a major release is required when there is an ABI
> breakage.

That's the usual practice, but not in numpy (neither in python BTW). The 
ABI is regularly broken in minor releases.

> Hopefully some users of the numpy ABI can provide some feedback on their needs.

Breaking the ABI simply means that every single package using the C API 
will have to be recompiled. This means for example that every 
windows/mac os x binary out there is broken (including scipy binaries).

It is particularly painful if you need to have some packages with numpy 
1.3.0 and some with 1.4.0.

The idea is that in the current state, keeping the ABI is incredibly 
difficult, so we would need to severly change the C code (in backward 
incompatible ways, i.e. it would require to break the A*P*I as well) to 
control those issues later.

Guido explicitly asked not to break compatibility while staying under 
py3k, so we should try to do it once numpy has been ported to py3k (e.g. 
if numpy 1.5 still is not py3k compatible, do a 1.6 before a 2.0 - 
iterate if necessary :) ).

David


More information about the NumPy-Discussion mailing list