Bug #102495 for Date-Calc: Mktime failing with dates

Tue Mar 03 16:17:57 2015 maurice [...] cpan.org - Ticket created

Subject:

Mktime failing with dates > 2038 even with 64-bit Perls

Even if the underlying Perl supports epoch timestamps past 2038, Date::Calc is still raising an exception. $ perl -v This is perl 5, version 18, subversion 1 (v5.18.1) built for x86_64-linux $ perl -MDate::Calc -E 'say Date::Calc->VERSION' 6.3 Test code that demonstrates the problem. Note that Time::Piece handles the timestamps fine. #!/usr/bin/env perl use strict; use Try::Tiny; use Time::Piece; use Date::Format qw/ time2str /; use Date::Calc qw/ Mktime /; $| = 1; my $t = localtime; my $year = $t->year; foreach (1..30) { $t = $t + (86400 * 365); print "T::P ", $t->ymd, "\n"; try { my $time = Mktime($t->year, 1, 1, 1, 1, 1); print "D::C ", time2str("%Y-%m-%d", $time), "\n"; } catch { warn "Date::Calc Mktime() failed: $_\n"; }; print "\n"; }

Wed Mar 04 16:22:43 2015 SREZIC [...] cpan.org - Correspondence added

On 2015-03-03 16:17:57, MAURICE wrote: Show quoted text

> Even if the underlying Perl supports epoch timestamps past 2038, > Date::Calc is still raising an exception. > > $ perl -v > This is perl 5, version 18, subversion 1 (v5.18.1) built for x86_64- > linux > > $ perl -MDate::Calc -E 'say Date::Calc->VERSION' > 6.3 > > Test code that demonstrates the problem. Note that Time::Piece handles > the timestamps fine. >

Both Date::Calc::PP and Date::Calc::XS are affected. It seems that things are working as expected if the y2038 restriction in Date::Calc::PP::DateCalc_mktime was removed, at least on a 64bit machine.

Wed Mar 04 16:22:44 2015 The RT System itself - Status changed from 'new' to 'open'

Fri Mar 06 18:04:13 2015 STBEY [...] cpan.org - Correspondence added

On Wed Mar 04 16:22:43 2015, SREZIC wrote: Show quoted text

> On 2015-03-03 16:17:57, MAURICE wrote:

> > Even if the underlying Perl supports epoch timestamps past 2038, > > Date::Calc is still raising an exception. > > > > $ perl -v > > This is perl 5, version 18, subversion 1 (v5.18.1) built for x86_64- > > linux > > > > $ perl -MDate::Calc -E 'say Date::Calc->VERSION' > > 6.3 > > > > Test code that demonstrates the problem. Note that Time::Piece > > handles > > the timestamps fine. > >

> > Both Date::Calc::PP and Date::Calc::XS are affected. > > It seems that things are working as expected if the y2038 restriction > in Date::Calc::PP::DateCalc_mktime was removed, at least on a 64bit > machine.

Mktime() is documented as not being very reliable ("USE AT YOUR OWN RISK!"), because it depends on the implementation on each machine and each Perl version, which I can't do anything about. So if it works for you, you are lucky, if it doesn't, you are out of luck. If you can find a workaround, the better for you. Sorry!

Fri Mar 06 19:38:58 2015 maurice [...] cpan.org - Correspondence added

On Fri Mar 06 18:04:13 2015, STBEY wrote: Show quoted text

> Mktime() is documented as not being very reliable ("USE AT YOUR OWN > RISK!"), > because it depends on the implementation on each machine and each Perl > version, > which I can't do anything about. > So if it works for you, you are lucky, if it doesn't, you are out of > luck. > If you can find a workaround, the better for you. > > Sorry!

I think the confusing thing is the docs suggest it's just a wrapper, and includes this note: 'A fatal "date out of range" error will occur if the given date cannot be expressed in terms of seconds since the epoch (this happens for instance when the date lies before the epoch, or if it is later than 19-Jan-2038 03:14:07 GMT on 32 bit Unix systems, or later than 06-Feb-2040 06:28:15 (local time) on a Macintosh with MacOS Classic). ' Which kind of suggests to me that the 32-bitness has something to do with the limit -- but it doesn't. Same failure on 64-bit, since the limit check is hard-coded. Would you consider a patch if I worked on one?

Fri Mar 06 20:06:19 2015 STBEY [...] cpan.org - Correspondence added

On Fri Mar 06 19:38:58 2015, MAURICE wrote: Show quoted text

> On Fri Mar 06 18:04:13 2015, STBEY wrote:

> > Mktime() is documented as not being very reliable ("USE AT YOUR OWN > > RISK!"), > > because it depends on the implementation on each machine and each > > Perl > > version, > > which I can't do anything about. > > So if it works for you, you are lucky, if it doesn't, you are out of > > luck. > > If you can find a workaround, the better for you. > > > > Sorry!

> > I think the confusing thing is the docs suggest it's just a wrapper, > and includes this note: > > 'A fatal "date out of range" error will occur if the given date cannot > be expressed in terms of seconds since the epoch (this happens for > instance when the date lies before the epoch, or if it is later than > 19-Jan-2038 03:14:07 GMT on 32 bit Unix systems, or later than > 06-Feb-2040 06:28:15 (local time) on a Macintosh with MacOS Classic). > ' > > Which kind of suggests to me that the 32-bitness has something to do > with the limit -- but it doesn't. Same failure on 64-bit, since the > limit check is hard-coded. > > Would you consider a patch if I worked on one?

Yes, the function Mktime() in my module is just a wrapper around whatever Perl/POSIX/C provide. This is why the results are so unpredictable and why I can't do anything about it. There is no code of mine whatsoever, I just pass the parameters on to the internal function and return its output - that's it. I doubt that your patch will be able to fix that, and even if it does, it may not do so on other machines with other OS's, on which it might fail catastrophically. Therefore I would rather not change anything in my code. These functions are only for the not faint of heart! Better alternatives are Date2Time() and Time2Date(), which are my own code - and they work.

Fri Mar 06 20:22:32 2015 maurice [...] cpan.org - Correspondence added

On Fri Mar 06 20:06:19 2015, STBEY wrote: Show quoted text

> Yes, the function Mktime() in my module is just a wrapper around > whatever Perl/POSIX/C provide. > This is why the results are so unpredictable and why I can't do > anything about it. > There is no code of mine whatsoever, I just pass the parameters on to > the internal function and return its output - that's it.

There's a bit of checking of the parameter values prior to passing them to the underlying mktime(). If I remove those checks, Mktime() works fine past 2038 on my 64-bit system. See the attached diff. I'm not suggesting this as a patch... just demonstrating that the limit is artificial. And I understand your suggestion of using something else, and I am... just would like others to not hit this problem if it's an artificial limitation. Alternatively, I could submit a patch to the docs that just says years > 2038 are not supported, regardless of 64-bit support.

Subject:

date-calc-diff.txt

diff -uBbr Date-Calc-6.3-orig/lib/Date/Calc/PP.pm Date-Calc-6.3-new/lib/Date/Calc/PP.pm --- Date-Calc-6.3-orig/lib/Date/Calc/PP.pm 2009-10-30 23:53:01.000000000 -0700 +++ Date-Calc-6.3-new/lib/Date/Calc/PP.pm 2015-03-06 17:16:09.276330580 -0800 @@ -2033,46 +2033,46 @@ { my($_seconds,$year,$month,$day,$hour,$min,$sec,$doy,$dow,$dst) = @_; $$_seconds = 0; - if ($^O eq 'MacOS') - { - return 0 if - ( - ($year < 1904) or ($year > 2040) or - ($month < 1) or ($month > 12) or - ($day < 1) or ($day > 31) or - ($hour < 0) or ($hour > 23) or - ($min < 0) or ($min > 59) or - ($sec < 0) or ($sec > 59) - ); - return 0 if - ( - ($year == 2040) and ( ($month > 2) or - ( ($month == 2) and ( ($day > 6) or - ( ($day == 6) and ( ($hour > 6) or - ( ($hour == 6) and ( ($min > 28) or - ( ($min == 28) and ($sec > 15) ) ))))))) - ); - } - else - { - return 0 if - ( - ($year < 1970) or ($year > 2038) or - ($month < 1) or ($month > 12) or - ($day < 1) or ($day > 31) or - ($hour < 0) or ($hour > 23) or - ($min < 0) or ($min > 59) or - ($sec < 0) or ($sec > 59) - ); - return 0 if - ( - ($year == 2038) and ( ($month > 1) or - ( ($month == 1) and ( ($day > 19) or - ( ($day == 19) and ( ($hour > 3) or - ( ($hour == 3) and ( ($min > 14) or - ( ($min == 14) and ($sec > 7) ) ))))))) - ); - } + # if ($^O eq 'MacOS') + # { + # return 0 if + # ( + # ($year < 1904) or ($year > 2040) or + # ($month < 1) or ($month > 12) or + # ($day < 1) or ($day > 31) or + # ($hour < 0) or ($hour > 23) or + # ($min < 0) or ($min > 59) or + # ($sec < 0) or ($sec > 59) + # ); + # return 0 if + # ( + # ($year == 2040) and ( ($month > 2) or + # ( ($month == 2) and ( ($day > 6) or + # ( ($day == 6) and ( ($hour > 6) or + # ( ($hour == 6) and ( ($min > 28) or + # ( ($min == 28) and ($sec > 15) ) ))))))) + # ); + # } + # else + # { + # return 0 if + # ( + # ($year < 1970) or ($year > 2038) or + # ($month < 1) or ($month > 12) or + # ($day < 1) or ($day > 31) or + # ($hour < 0) or ($hour > 23) or + # ($min < 0) or ($min > 59) or + # ($sec < 0) or ($sec > 59) + # ); + # return 0 if + # ( + # ($year == 2038) and ( ($month > 1) or + # ( ($month == 1) and ( ($day > 19) or + # ( ($day == 19) and ( ($hour > 3) or + # ( ($hour == 3) and ( ($min > 14) or + # ( ($min == 14) and ($sec > 7) ) ))))))) + # ); + # } $year -= 1900; $month--; if ($doy <= 0) { $doy = -1; } Only in Date-Calc-6.3-new/: try

Sat Mar 07 10:20:43 2015 STBEY [...] cpan.org - Correspondence added

On Fri Mar 06 20:22:32 2015, MAURICE wrote: Show quoted text

> On Fri Mar 06 20:06:19 2015, STBEY wrote:

> > Yes, the function Mktime() in my module is just a wrapper around > > whatever Perl/POSIX/C provide. > > This is why the results are so unpredictable and why I can't do > > anything about it. > > There is no code of mine whatsoever, I just pass the parameters on to > > the internal function and return its output - that's it.

> > There's a bit of checking of the parameter values prior to passing > them to the underlying mktime(). If I remove those checks, Mktime() > works fine past 2038 on my 64-bit system. > > See the attached diff. I'm not suggesting this as a patch... just > demonstrating that the limit is artificial. > > And I understand your suggestion of using something else, and I am... > just would like others to not hit this problem if it's an artificial > limitation. > > Alternatively, I could submit a patch to the docs that just says years

> > 2038 are not supported, regardless of 64-bit support.

I added those checks in order to avoid problems with people complaining that the functions don't work (when in reality they fed them wrong parameters). I wanted to minimize my amount of trouble. :-) Of course you are free to remove the checks and this might work perfectly for you and on your machine. However, there is no guarantee that this will work equally well for somebody else and on a different machine. That's my concern.

Sat Mar 07 14:30:20 2015 SREZIC [...] cpan.org - Correspondence added

On 2015-03-07 10:20:43, STBEY wrote: Show quoted text

> On Fri Mar 06 20:22:32 2015, MAURICE wrote:

> > On Fri Mar 06 20:06:19 2015, STBEY wrote:

> > > Yes, the function Mktime() in my module is just a wrapper around > > > whatever Perl/POSIX/C provide. > > > This is why the results are so unpredictable and why I can't do > > > anything about it. > > > There is no code of mine whatsoever, I just pass the parameters on > > > to > > > the internal function and return its output - that's it.

> > > > There's a bit of checking of the parameter values prior to passing > > them to the underlying mktime(). If I remove those checks, Mktime() > > works fine past 2038 on my 64-bit system. > > > > See the attached diff. I'm not suggesting this as a patch... just > > demonstrating that the limit is artificial. > > > > And I understand your suggestion of using something else, and I am... > > just would like others to not hit this problem if it's an artificial > > limitation. > > > > Alternatively, I could submit a patch to the docs that just says > > years

> > > 2038 are not supported, regardless of 64-bit support.

> > I added those checks in order to avoid problems with people > complaining that the functions don't work (when in reality they fed > them wrong parameters). I wanted to minimize my amount of trouble. :-) > > Of course you are free to remove the checks and this might work > perfectly for you and on your machine. > However, there is no guarantee that this will work equally well for > somebody else and on a different machine. > That's my concern.

How about a whitelist? So far it seems to work on linux and freebsd systems, at least on 64bit machines. We have cpantesters to detect if it works on other platforms, too. Regards, Slaven

Sat Mar 07 14:36:25 2015 STBEY [...] cpan.org - Correspondence added

On Sat Mar 07 14:30:20 2015, SREZIC wrote: Show quoted text

> On 2015-03-07 10:20:43, STBEY wrote:

> > On Fri Mar 06 20:22:32 2015, MAURICE wrote:

> > > On Fri Mar 06 20:06:19 2015, STBEY wrote:

> > > > Yes, the function Mktime() in my module is just a wrapper around > > > > whatever Perl/POSIX/C provide. > > > > This is why the results are so unpredictable and why I can't do > > > > anything about it. > > > > There is no code of mine whatsoever, I just pass the parameters > > > > on > > > > to > > > > the internal function and return its output - that's it.

> > > > > > There's a bit of checking of the parameter values prior to passing > > > them to the underlying mktime(). If I remove those checks, Mktime() > > > works fine past 2038 on my 64-bit system. > > > > > > See the attached diff. I'm not suggesting this as a patch... just > > > demonstrating that the limit is artificial. > > > > > > And I understand your suggestion of using something else, and I > > > am... > > > just would like others to not hit this problem if it's an > > > artificial > > > limitation. > > > > > > Alternatively, I could submit a patch to the docs that just says > > > years

> > > > 2038 are not supported, regardless of 64-bit support.

> > > > I added those checks in order to avoid problems with people > > complaining that the functions don't work (when in reality they fed > > them wrong parameters). I wanted to minimize my amount of trouble. :- > > ) > > > > Of course you are free to remove the checks and this might work > > perfectly for you and on your machine. > > However, there is no guarantee that this will work equally well for > > somebody else and on a different machine. > > That's my concern.

> > How about a whitelist? So far it seems to work on linux and freebsd > systems, at least on 64bit machines. We have cpantesters to detect if > it works on other platforms, too. > > Regards, > Slaven

That may be a possibility, but I don't have the time anymore to do development or major maintenance on this module. I can only do the bare minimum to keep the module working and the tests from failing.

Bug #102495 for Date-Calc: Mktime failing with dates > 2038 even with 64-bit Perls