Bug #93095 for Time-Piece: strptime/strftime issues with %z and %Z

Mon Feb 17 15:38:47 2014 gelbman [...] gmail.com - Ticket created

Subject:

strptime issues with %z and %Z

When I use strptime to parse a date with a timezone offset like -0600 the resulting time is offset +6 hours from the local timezone when it should be offset from GMT. For example: #!/usr/bin/perl use Time::Piece; # prints 1984-01-01 05:00 UTC +0000 # as it should $ENV{TZ} = "UTC"; my $t = Time::Piece->strptime("1984-01-01 00:00 -0500", "%Y-%m-%d %H:%M %z"); print $t->strftime("%Y-%m-%d %H:%M %Z %z\n"); # prints 1984-01-01 05:00 CST -0600 # but should be 1983-12-31 23:00 CST -0600 $ENV{TZ} = "CST6CDT"; my $t = Time::Piece->strptime("1984-01-01 00:00 -0500", "%Y-%m-%d %H:%M %z"); print $t->strftime("%Y-%m-%d %H:%M %Z %z\n"); Also %Z seems completely broken: # Error parsing time at /Library/Perl/5.16/darwin-thread-multi-2level/Time/Piece.pm line 469. $ENV{TZ} = "CST6CDT"; my $t = Time::Piece->strptime("1984-01-01 00:00 EST", "%Y-%m-%d %H:%M %Z"); print $t->strftime("%Y-%m-%d %H:%M %Z %z\n");

Tue Jul 29 15:59:14 2014 tsibley [...] cpan.org - Requestor TSIBLEY added

Tue Jul 29 16:07:21 2014 tsibley [...] cpan.org - Subject changed from 'strptime issues with %z and %Z' to 'strptime/strftime issues with %z and %Z'

Tue Jul 29 16:21:33 2014 tsibley [...] cpan.org - Forwarded Transaction #1327654 to tsibley@cpan.org

Tue Jul 29 18:54:18 2014 tsibley [...] cpan.org - Correspondence added

Subject:	Re: [rt.cpan.org #93095] strptime issues with %z and %Z
Date:	Tue, 29 Jul 2014 18:53:48 -0400
To:	bug-Time-Piece [...] rt.cpan.org
From:	Thomas Sibley <tsibley [...] cpan.org>

This behaviour bit me recently, and when I went digging I learned more than I really wanted to know (as is always the case with date times). I summarize below... Show quoted text

> When I use strptime to parse a date with a timezone offset like -0600 > the resulting time is offset +6 hours from the local timezone when it > should be offset from GMT. > > [...] > > # prints 1984-01-01 05:00 CST -0600 > # but should be 1983-12-31 23:00 CST -0600 > $ENV{TZ} = "CST6CDT"; > my $t = Time::Piece->strptime("1984-01-01 00:00 -0500", "%Y-%m-%d %H:%M %z"); > print $t->strftime("%Y-%m-%d %H:%M %Z %z\n");

I believe the problem you're seeing is in formatting, not parsing. Your examples touch both strptime and strftime, which I think leads to some confusion over where the fault lies. It'd be good to break the examples into tests that only exercise one function at a time and show the failures more clearly. The formatting bug is that Time::Piece doesn't actually pass any time zone offset information to the underlying strftime XS/C function it calls, so these parts of the C struct tm are initialized to the machine's localtime (usually via init_tm provided by Perl's util.c) [1]. Hence the time values are in GMT but the local zone (%z or %Z) is returned without any conversion being done. When the format contains %z or %Z, strftime needs to either a) convert to local time first (from GMT or whatever offset is in the object) or b) set the appropriate offsets and name in the C struct so a zone other than the local time is reported. All that said, note that the strptime provided by Time::Piece ignores the got_GMT flag it gets from the parser [2]. The effect of this is that Time::Piece always treats strings without %z as GMT, not the local zone. This may be unintuitive and certainly should be documented. Strings *with* %z are converted to GMT while parsing, so if you're explicit, such as in your examples, you're OK. Show quoted text

> Also %Z seems completely broken:

Real strptime support for %Z is pretty hard to find, including in glibc. Time::Piece's strptime is no different, and it only recogizes "GMT" [3]. It is worth noting that GNU date(1) can handle lots of timezone specifications (refer to the parse_datetime function in GNU coreutils), though it does recommend against ambiguous names like "EST" [4]. [1] http://perl5.git.perl.org/perl.git/blob/HEAD:/util.c#l3599 [2] https://metacpan.org/source/RJBS/Time-Piece-1.27/Piece.xs#L1107 [3] https://metacpan.org/source/RJBS/Time-Piece-1.27/Piece.xs#L894 [4] https://www.gnu.org/software/coreutils/manual/html_node/Time-zone-items.html#Time-zone-items

Sat Apr 04 16:07:31 2015 ESAYM [...] cpan.org - Correspondence added

On Mon Feb 17 15:38:47 2014, iamjake wrote: Show quoted text

> When I use strptime to parse a date with a timezone offset like -0600 > the resulting time is offset +6 hours from the local timezone when it > should be offset from GMT. > > For example: > > #!/usr/bin/perl > use Time::Piece; > > # prints 1984-01-01 05:00 UTC +0000 > # as it should > $ENV{TZ} = "UTC"; > my $t = Time::Piece->strptime("1984-01-01 00:00 -0500", "%Y-%m-%d > %H:%M %z"); > print $t->strftime("%Y-%m-%d %H:%M %Z %z\n"); > > # prints 1984-01-01 05:00 CST -0600 > # but should be 1983-12-31 23:00 CST -0600 > $ENV{TZ} = "CST6CDT"; > my $t = Time::Piece->strptime("1984-01-01 00:00 -0500", "%Y-%m-%d > %H:%M %z"); > print $t->strftime("%Y-%m-%d %H:%M %Z %z\n"); > > Also %Z seems completely broken: > > # Error parsing time at /Library/Perl/5.16/darwin-thread-multi- > 2level/Time/Piece.pm line 469. > $ENV{TZ} = "CST6CDT"; > my $t = Time::Piece->strptime("1984-01-01 00:00 EST", "%Y-%m-%d %H:%M > %Z"); > print $t->strftime("%Y-%m-%d %H:%M %Z %z\n");

Strangely, using the native functions provided in libc shows the behavior is completely different. Converting timezones doesn't seem to really be supported. Still investigating, trying to get an example to work in pure C before I tackle the _strptime in xs. It might be converting to the epoch might be the best way forward but I'll see if there is another way.

Sat Apr 04 16:07:32 2015 The RT System itself - Status changed from 'new' to 'open'

Sat Apr 04 16:09:01 2015 ESAYM [...] cpan.org - Correspondence added

On Sat Apr 04 16:07:31 2015, ESAYM wrote: Show quoted text

> On Mon Feb 17 15:38:47 2014, iamjake wrote:

> > When I use strptime to parse a date with a timezone offset like -0600 > > the resulting time is offset +6 hours from the local timezone when it > > should be offset from GMT. > > > > For example: > > > > #!/usr/bin/perl > > use Time::Piece; > > > > # prints 1984-01-01 05:00 UTC +0000 > > # as it should > > $ENV{TZ} = "UTC"; > > my $t = Time::Piece->strptime("1984-01-01 00:00 -0500", "%Y-%m-%d > > %H:%M %z"); > > print $t->strftime("%Y-%m-%d %H:%M %Z %z\n"); > > > > # prints 1984-01-01 05:00 CST -0600 > > # but should be 1983-12-31 23:00 CST -0600 > > $ENV{TZ} = "CST6CDT"; > > my $t = Time::Piece->strptime("1984-01-01 00:00 -0500", "%Y-%m-%d > > %H:%M %z"); > > print $t->strftime("%Y-%m-%d %H:%M %Z %z\n"); > > > > Also %Z seems completely broken: > > > > # Error parsing time at /Library/Perl/5.16/darwin-thread-multi- > > 2level/Time/Piece.pm line 469. > > $ENV{TZ} = "CST6CDT"; > > my $t = Time::Piece->strptime("1984-01-01 00:00 EST", "%Y-%m-%d %H:%M > > %Z"); > > print $t->strftime("%Y-%m-%d %H:%M %Z %z\n");

> > > > Strangely, using the native functions provided in libc shows the > behavior is completely different. Converting timezones doesn't seem to > really be supported. Still investigating, trying to get an example to > work in pure C before I tackle the _strptime in xs. It might be > converting to the epoch might be the best way forward but I'll see if > there is another way.

I might add my above comment was only concerning how strptime builds the tm struct, strftime is a different beast...

Mon Apr 06 22:07:03 2015 rjbs [...] cpan.org - Correspondence added

Something, possibly related to this, has broken String-Errf: http://www.cpantesters.org/cpan/report/cae9ca10-dcad-11e4-b7a8-a446e14af301 -- rjbs

Tue Apr 07 14:59:18 2015 ESAYM [...] cpan.org - Requestor ESAYM added

Tue Apr 07 17:26:45 2015 ESAYM [...] cpan.org - Correspondence added

On Mon Apr 06 22:07:03 2015, RJBS wrote: Show quoted text

> Something, possibly related to this, has broken String-Errf: > > http://www.cpantesters.org/cpan/report/cae9ca10-dcad-11e4-b7a8-a446e14af301

Indeed, thank you for the update. The issue stems from my call to native mktime. This is done to fix issues where dates in the past would be given the wrong dst timezone in the output of %z in strftime. I demonstrate this in the attached file, the timezone is wrong because it was initially set (In Piece.xs) by a call to the native libc localtime which populates the tmzone struct element with the current zone regardless if the broken down time is from a different dst. Let me put some thought into this and commit a patch against 1.29_03.

Subject:

time_test.pl

#! /usr/bin/env perl use strict; use Time::Piece; use POSIX qw(strftime); my $t = localtime(951827696); my $tt = gmtime(951827696); my @l = localtime(951827696); my @ll = gmtime(951827696); print $t->strftime("%F %T %Z %z") . "\n"; print $tt->strftime("%F %T %Z %z") . "\n"; print strftime("%F %T %Z %z", @l) . "\n"; print strftime("%F %T %Z %z", @ll) . "\n"; #output: #2000-02-29 06:34:56 CDT -0500 #2000-02-29 12:34:56 CDT -0500 #2000-02-29 06:34:56 CST -0600 #2000-02-29 12:34:56 CST -0600

Fri Apr 10 13:06:20 2015 ESAYM [...] cpan.org - Correspondence added

On Mon Apr 06 22:07:03 2015, RJBS wrote: Show quoted text

> Something, possibly related to this, has broken String-Errf: > > http://www.cpantesters.org/cpan/report/cae9ca10-dcad-11e4-b7a8-a446e14af301

This is fixed in 1.29_03, but I am now toying with the idea of just passing in the epoch to _strftime in the xs code. I think that would be simpler in the end and one wouldn't need to worry at all about platform differences with the tm struct. Also added more test cases and I have a simple script that can pull and build all the reverse dependencies of Time::Piece so I shouldn't be breaking other modules again :)

Mon Jun 19 16:28:17 2017 ESAYM [...] cpan.org - Correspondence added

On Mon Feb 17 15:38:47 2014, iamjake wrote: Show quoted text

> When I use strptime to parse a date with a timezone offset like -0600 > the resulting time is offset +6 hours from the local timezone when it > should be offset from GMT. > > For example: > > #!/usr/bin/perl > use Time::Piece; > > # prints 1984-01-01 05:00 UTC +0000 > # as it should > $ENV{TZ} = "UTC"; > my $t = Time::Piece->strptime("1984-01-01 00:00 -0500", "%Y-%m-%d > %H:%M %z"); > print $t->strftime("%Y-%m-%d %H:%M %Z %z\n"); > > # prints 1984-01-01 05:00 CST -0600 > # but should be 1983-12-31 23:00 CST -0600 > $ENV{TZ} = "CST6CDT"; > my $t = Time::Piece->strptime("1984-01-01 00:00 -0500", "%Y-%m-%d > %H:%M %z"); > print $t->strftime("%Y-%m-%d %H:%M %Z %z\n"); > > Also %Z seems completely broken: > > # Error parsing time at /Library/Perl/5.16/darwin-thread-multi- > 2level/Time/Piece.pm line 469. > $ENV{TZ} = "CST6CDT"; > my $t = Time::Piece->strptime("1984-01-01 00:00 EST", "%Y-%m-%d %H:%M > %Z"); > print $t->strftime("%Y-%m-%d %H:%M %Z %z\n");

There seems to have been some confusion on my part here (and in other places) Calling like this: Time::Piece->strptime() defaults to returning a TP instance of GMT Getting a local TP instance via localtime(), is different. So of course doing something like: Show quoted text

>my $t = Time::Piece->strptime("1984-01-01 00:00 EST", "%Y-%m-%d %H:%M >print $t->strftime("%Y-%m-%d %H:%M %Z %z\n");

will give you a GMT time (as $t->[c_islocal] will be set to 1). So this problem was made out to be a strftime issue, when it is really a strptime issue.