Bug #58159 for Date-Manip: Ambiguous timezone "EST" handled differently in v5 and v6

Sat Jun 05 06:53:15 2010 chrisb [...] debian.org - Ticket created

Subject:

Ambiguous timezone "EST" handled differently in v5 and v6

This is related to a bug reported in Debian against the DateTime::Format::DateManip perl module: http://bugs.debian.org/cgi- bin/bugreport.cgi?bug=574317 The test suite for that module has started failing with Date::Manip v6, because it uses the timezone "EST" in the test. The new timezone code in version 6 treats this timezone differently from version 5. It comes down to the fact that there's more than one timezone with the abbreviation "EST": EST Eastern Summer Time Australia UTC + 11 hours EST Eastern Standard Time Australia UTC + 10 hours EST Eastern Standard Time North America UTC - 5 hours (from: http://www.timeanddate.com/library/abbreviations/timezones/) However, from the wikipedia article on Australian Eastern Standard Time: The proper names of Australia's time zones vary. In international contexts they are often called Australian Western Standard Time (AWST), Australian Central Standard Time (ACST), and Australian Eastern Standard Time (AEST). In domestic contexts the leading "Australian" is often dropped. Version 5 used to pick the American EST, whereas version 6 picks the Australian version. I think that this change of behaviour is a bug. It certainly looks to me as if the American EST is more widely used than the Australian. Perhaps Date::Manip should have some way to assign priorities to these ambiguous timezones? The default behaviour would then be to treat the abbreviation EST as the American one, with some way to adjust the priorities if necessary..

Sat Jun 05 07:27:52 2010 sbeck [...] cpan.org - Correspondence added

What you are reporting is not a bug in v6... it's actually a bug in v5 that is now fixed. Date::Manip v5 treated timezone abbreviations (e.g. EST) as timezones, so it treated EST the same way as the timezone America/New_York. Clearly, this is NOT the correct behavior, and v6 finally corrects that. V6 uses the abbreviations to determine a timezone in a way that produces a correct date. In other words, in the America/New_York timezone, the abbreviation on Jun 5 is EDT, so the date 'Jun 5 EST' is NOT in the America/New_York timezone. It is either in some other timezone (if there is a timezone that uses the EST abbreviation on Jun 5) or invalid. In this case, it falls in an Australian timezone, and that's exactly what is happening in the "bug" you're reporting. The solution isn't to change Date::Manip's behavior (which is correct after many years of being wrong)... it's to rewrite the test. With respect to your comment about the wikipedia description... all of the timezone information in Date::Manip comes from recognized standards. The bulk of it comes from the Olsen database which is pretty much the definitive source of timezone information. Wikipedia is nice... but if it doesn't match the information in the Olsen database, I'm going to have to treat wikipedia as being the incorrect one. NOTE: I have NOT investigated the actual results of the failed test, so I'm not sure whether the information on wikipedia is incorrect, or if you're not looking up the correct information (i.e. comparing apples to oranges). Finally, with respect to your suggestion that Date::Manip have a way to prioritize ambiguous timezones, it does. These are documented in the Date::Manip::TZ manual. Look at the define_abbrev, define_alias, and define_offset methods. However, these will NOT solve the 'problem' you are reporting, because no prioritization will ever cause Date::Manip to produce an invalid date. In other words, no matter what priority you assign, 'Jun 5 EST' is NEVER going to be in America/New_York. I hope this clarifies things.

Sat Jun 05 07:27:53 2010 The RT System itself - Status changed from 'new' to 'open'

Sat Jun 05 07:27:53 2010 sbeck [...] cpan.org - Status changed from 'open' to 'resolved'

Thu Jun 10 14:59:00 2010 chrisb [...] debian.org - Correspondence added

From:

chrisb [...] debian.org

On Sat Jun 05 07:27:52 2010, SBECK wrote: Show quoted text

> What you are reporting is not a bug in v6... it's actually a bug in v5 > that is now fixed. > > Date::Manip v5 treated timezone abbreviations (e.g. EST) as timezones, > so it treated EST the same way as the timezone America/New_York. > Clearly, this is NOT the correct behavior, and v6 finally corrects

that. Show quoted text

> V6 uses the abbreviations to determine a timezone in a way that

produces Show quoted text

> a correct date. In other words, in the America/New_York timezone, the > abbreviation on Jun 5 is EDT, so the date 'Jun 5 EST' is NOT in the > America/New_York timezone. It is either in some other timezone (if

there Show quoted text

> is a timezone that uses the EST abbreviation on Jun 5) or invalid.

Sorry, it seems I neglected to include the date in the original report. Actually, the date string that we're trying to parse is "March 23, 2003 12:00 EST", which seems to be during winter time (EST) according to 'date': $ TZ='America/New_York' date --date '2003-03-23 12:00' Sun Mar 23 12:00:00 EST 2003 Show quoted text

> Finally, with respect to your suggestion that Date::Manip have a > way to prioritize ambiguous timezones, it does. These are > documented in the Date::Manip::TZ manual. Look at the > define_abbrev, define_alias, and define_offset methods.

In which case, I believe that "EST" during winter time should pick America over Australia. As per the discussion on the Debian bug report, we believe that "EST" is more commonly used to describe the American timezone than the Australian, so where it is ambiguous, it should prefer the American timezone. This would give Date::Manip the best chance of correctly parsing an ambiguous date.

Thu Jun 10 14:59:00 2010 The RT System itself - Status changed from 'resolved' to 'open'

Thu Jun 10 15:38:42 2010 sbeck [...] cpan.org - Correspondence added

Show quoted text

> Sorry, it seems I neglected to include the date in the original report. > Actually, the date string that we're trying to parse is "March 23, 2003 > 12:00 EST", which seems to be during winter time (EST) according to > 'date':

I did some more looking... previously, I hadn't actually looked at the test script, since it's not part of Date::Manip. It is doing: Date_Init("TZ=EST"); instead of the much better: Date_Init("TZ=America/New_York"); Actually, since this is deprecated, an even better call would be: Date_Init("setdate=now,America/New_York"); Since they are specifying the timezone using the only the abbreviation, it's using some information from the system, including the current timezone value of isdst. Since it is currently in DST, it's ending up using the Australian timezone. In other words, the proper way to specify a timezone is to use an actual timezone name/alias. EST is kind of used as a timezone alias in Date::Manip 5.xx, but that is only because timezone support was so broken in 5.xx. Show quoted text

> > Finally, with respect to your suggestion that Date::Manip have a > > way to prioritize ambiguous timezones, it does. These are > > documented in the Date::Manip::TZ manual. Look at the > > define_abbrev, define_alias, and define_offset methods.

> > In which case, I believe that "EST" during winter time should pick > America over Australia. As per the discussion on the Debian bug report, > we believe that "EST" is more commonly used to describe the American > timezone than the Australian, so where it is ambiguous, it should prefer > the American timezone. This would give Date::Manip the best chance of > correctly parsing an ambiguous date.

And it does... but this is done in the Date_Init call, NOT when the date is parsed. So, the timezone was set before a date of 'March 23' was used. In any case,this is actually kind of a tricky problem since there are now two versions of Date::Manip that are not 100% compatible, and if the DateTime-Format-DateManip module wants their test suite to work with either one, they're going to have to add some version-specific handling in a few cases. It shouldn't be hard, and I'll be happy to work with them if they are interested.

Thu Jun 10 15:39:39 2010 sbeck [...] cpan.org - Status changed from 'open' to 'resolved'