[Numpy-discussion] py3 datetime woes (was Re: Code Freeze for NumPy 1.7)

Nathaniel Smith njs@pobox....
Mon Jul 16 08:27:06 CDT 2012


On Sun, Jul 15, 2012 at 5:32 PM, Ralf Gommers
<ralf.gommers@googlemail.com> wrote:
>
>
> On Sun, Jul 15, 2012 at 5:57 PM, Nathaniel Smith <njs@pobox.com> wrote:
>>
>> On Sun, Jul 15, 2012 at 1:08 PM, Ralf Gommers
>> <ralf.gommers@googlemail.com> wrote:
>> >
>> >
>> > On Sun, Jul 15, 2012 at 12:45 AM, Travis Oliphant <travis@continuum.io>
>> > wrote:
>> >>
>> >>
>> >> Hey all,
>> >>
>> >> We are nearing a code-freeze for NumPy 1.7.   Are there any last-minute
>> >> changes people are wanting to push into NumPy 1.7?  We should discuss
>> >> them
>> >> as soon as possible.
>> >>
>> >> I'm proposing a code-freeze at midnight UTC on July 18th (7:00pm CDT on
>> >> July 17th).   This will allow the creation of beta releases of NumPy on
>> >> the
>> >> 18th of July. This is a few days later than originally hoped for ---
>> >> largely
>> >> due to unexpected travel schedules of Ondrej and I, but it does give
>> >> people
>> >> a few more days to get patches in.  Of course, we will be able to apply
>> >> bug-fixes to the 1.7.x branch once the tag is made.
>> >
>> >
>> > What about the tickets still open for 1.7.0
>> > (http://projects.scipy.org/numpy/report/3)? There are a few important
>> > ones
>> > left.
>> >
>> > These I would consider blockers:
>> >   - #2108 Datetime failures with MinGW
>>
>> Is there a description anywhere of what the problem actually is here?
>> I looked at the ticket, which referred to a PR, and it's hard to work
>> out from the PR discussion what the actual remaining test failures are
>> -- and there definitely doesn't seem to be any description of the
>> underlying problem. (Something about working 64-bit time_t on windows
>> being difficult depending on the compiler used?)
>
> There's a lot more discussion on
> http://projects.scipy.org/numpy/ticket/1909
> https://github.com/numpy/numpy/pull/156
> https://github.com/numpy/numpy/pull/161.
>
> The issue is that for MinGW 3.x some _s / _t functions seem to be missing.
> And we don't yet support MinGW 4.x.
>
> Current issues can be seen from the last test log on our Windows XP buildbot
> (June 29,
> http://buildbot.scipy.org/builders/Windows_XP_x86/builds/1124/steps/shell_1/logs/stdio):
>
> ======================================================================
> ERROR: test_datetime_arange (test_datetime.TestDateTime)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "C:\buildbot\numpy\b11\numpy-install\Lib\site-packages\numpy\core\tests\test_datetime.py",
> line 1351, in test_datetime_arange
>     assert_raises(ValueError, np.arange, np.datetime64('today'),
> OSError: Failed to use '_localtime64_s' to convert to a local time

...I don't understand how this is even building if the functions are missing.

Well, anyway.

MSVC provides both _s and regular versions of localtime and friends.
Mingw only provides the regular version. It looks like there are two
difference between the regular and _s versions of these functions:
1) The _s version reports errors by calling the "invalid parameter
handler" instead of just returning an error code. (I.e., by default,
if you try to work with a date that's out of range, then your program
will just abort(). Very friendly.)[1]
2) MSVC whines if you use the regular version

Basically AFAICT this is an interface designed to make it easier for
lousy programmers to write broken code that will still limp along more
successfully than it otherwise would. There's a legitimate role for
such interfaces, but I'm not sure numpy is it -- I think we can write
actual error handling code? The classic localtime() interface AFAICT
works perfectly -- it's even threadsafe, unlike most unix versions
[2].

I think the best solution here is to just switch to using localtime()
everywhere, and tell MSVC to shut up by using a #pragma. Here's an
example of how to do this taken from the Boost developer guidelines
[3]:

#if defined(_MSC_VER)
#pragma warning(push) // Save warning settings.
#pragma warning(disable : 4996) // Disable deprecated localtime/gmtime warning.
#endif

...

#if defined(_MSC_VER)
#pragma warning(pop) // Restore warnings to previous state.
#endif

(Yes, Boost's canonical example of how you control warnings on MSVC is
specifically for letting you use localtime() in piece -- I didn't add
that. Of course this could be hidden in a macro or helper function...
npy_localtime() or something.) Or alternatively one can just #define
_CRT_SECURE_NO_DEPRECATE to kill this warning in general. The Boost
developer's general advice on this particular warning is "Unless you
strongly believe that the 'secure' versions are useful, suppress."[3]

[1] http://msdn.microsoft.com/en-us/library/a442x3ye%28v=vs.80%29.aspx
, and note the "community content" at the bottom.
[2] http://old.nabble.com/Re%3A-Need-localtime_s-p13088250.html
[3] https://svn.boost.org/trac/boost/wiki/Guidelines/WarningsGuidelines

-n


More information about the NumPy-Discussion mailing list