[Numpy-discussion] Removing datetime support for 1.4.x series ?

David Cournapeau cournape@gmail....
Thu Feb 4 09:40:29 CST 2010


On Thu, Feb 4, 2010 at 4:37 PM, Travis Oliphant <oliphant@enthought.com> wrote:
>
> On Feb 4, 2010, at 12:59 AM, Charles R Harris wrote:
>
>
> On Wed, Feb 3, 2010 at 11:48 PM, Travis Oliphant <oliphant@enthought.com>
> wrote:
>>
>> On Feb 2, 2010, at 11:59 PM, David Cournapeau wrote:
>>
>> Travis Oliphant wrote:
>>
>> On Feb 2, 2010, at 11:46 PM, David Cournapeau wrote:
>>
>> On Wed, Feb 3, 2010 at 12:23 PM, David Cournapeau
>>
>> <david@silveregg.co.jp <mailto:david@silveregg.co.jp>> wrote:
>>
>>
>> Sorry, my question was badly worded: besides the metadata pointer, is
>>
>> there any other change related to the metadata infratructure which may
>>
>> potentially change changes the publicly exported structures ? I wonder
>>
>> whereas the metadata infrastructure can be kept in 1.4.x independently
>>
>> of the datetime support without breaking the ABI
>>
>> FWIW, keeping the metadata pointer, and only removing datetime-related
>>
>> things makes numpy 1.4.x backward compatible, at least as far as scipy
>>
>> is concerned. So it seems the PyArray_Funcs change is the only
>>
>> ABI-incompatible change.
>>
>> What do you mean by the "PyArray_Funcs change"?
>>
>> The change that broke the ABI is in the PyArray_Funcs structure
>> (ndarrayobject.h):
>>
>> struct {
>>         PyArray_VectorUnaryFunc *cast[NPY_NTYPES];
>>         ....
>>
>> Because NPY_NTYPES is bigger after the datetime change.
>>
>> If there is a way to have the datetime not expanding NPY_NTYPES, then I
>> think we can keep the ABI. I tried something with datetimes considered
>> as user types, but did not go very far (most certainly because I have
>> never used this part of the code before).
>>
>> Thanks for reminding me what the ABI problem is.  Yes, that will break it
>> (I was very suspicious that we could change the number of basic types
>> without ABI consequence but didn't have time to think about the real
>> problem).
>> My intention in adding the datetime data-type was not to try and preserve
>> ABI in the process.
>
> If so, then it would have been better to have been upfront about that when
> it went in. I know I pushed for inclusion, but I was told that the ABI could
> be preserved.

There may have been some miscommunication - in my mind, adding
datetime support so late was only possible under the condition that it
would not break the ABI. In particular, I have spent many hours
refactoring C code between 1.3.0 and 1.4.0, and a lot of that time was
kept to ensure I did not break the ABI, time which has been wasted. I
would also note that I was reluctant to add datetime so late because I
was precisely afraid something like that would happen.

In the grand scheme of things, it does not matter so much, we all have
different timelines and schedules, and I certainly don't think there
was any malicious intent from anyone to break things :) But I would
like to improve our development process so that we don't repeat the
same mistakes.

> Perhaps one way to articulate my perspective is the following:
> There are currently 2 groups of NumPy users:
>  1)  those who have re-compiled all of their code for 1.4.0
>  2)  those who haven't
> Group 1) will have to re-compile again no matter what we do (because we are
> either going to have to bump the ABI number or back-pedal).
> Group 2) will not have to re-compile once the new release comes out.
> I don't want to make Group 1) have to re-compile yet a third time when
> date-time support finally comes out.  If they have bitten the bullet now,
> they will be rewarded with a stable ABI (that will eventually have the
> benefit of better ufunc support for record arrays as well as the date-time
> features).

I think Group 1 is a negligible epsilon of Group 2, and moreover,
Group 1 is the most likely to be able to deal with those issues.

cheers,

David


More information about the NumPy-Discussion mailing list