Bug #103142 for Date-Manip: ParseDate() unexpected behavior with certain partial date constructs

Fri Mar 27 09:21:03 2015 gdg [...] zplane.com - Ticket created

Subject:	ParseDate() unexpected behavior with certain partial date constructs
Date:	Fri, 27 Mar 2015 07:20:45 -0600
To:	Sullivan Beck via RT <bug-Date-Manip [...] rt.cpan.org>
From:	Glenn Golden <gdg [...] zplane.com>

The following examples were prepared with Date::Manip::DM6 version "6.49 [america/denver]", running under perl 5.20.2. (The ParseDate() results are shown as-is except for an extra space inserted between date and time just for clarity.) Input ParseDate result ========= ================= 2015-06 20150601 00:00:00 # As expected: DoM defaults to 01 June 2015 20150620 00:00:00 # DoM inconsistent w/above. Also, DoM seems # to be taken from the first 2 digits of year. # Year is correct. June 1712 20120617 00:00:00 # Like above, again suggesting that DoM seems # to be taken from first 2 digits of year, # but now year is wrong too. 2095-06 20950601 00:00:00 # As expected: DoM defaults to 01, June 2095 19950620 00:00:00 # Again, DoM seems to be taken from first 2 # digits of year, and year is also wrong.

Fri Mar 27 13:45:37 2015 sbeck [...] cpan.org - Correspondence added

In parsing dates, I always place parsing a complete date (i.e. one where all of the values are supplied) over incomplete ones. As a matter of fact, I wouldn't even support partial dates except that ISO 8601 explicitly allows them. The ISO date 2015-06 contains no ambiguity. There is no way that a complete date can be extracted from it, so we can safely fall back to partial dates. Date::Manip supports the following documented formats: mmmD mmmDDYY mmmDDYYYY so you can say Jun 05 or June 052015, both of which are clear. The question is: what does Jun 2015' mean? The docs state that this is the equivalent of June 20, 2015, and I'm going to stick with that since mmmDDYY is complete whereas mmmYYYY is incomplete. I realize that that isn't what you want, but it's consistent with the goal of working with complete dates, and the documented formats. Also, the format is consistent within this set of supported formats (though I can definitely see how you can say that it is inconsistent with the ISO 8601 formats). I could add another function (like ParseIncompleteDate) which supported a bunch of incomplete date strings, but I'm not sure that this would really clarify the situation.

Fri Mar 27 13:45:37 2015 The RT System itself - Status changed from 'new' to 'open'

Fri Mar 27 13:45:39 2015 sbeck [...] cpan.org - Status changed from 'open' to 'resolved'

Sun Mar 29 14:37:53 2015 gdg [...] zplane.com - Correspondence added

Subject:	Re: [rt.cpan.org #103142] ParseDate() unexpected behavior with certain partial date constructs
Date:	Sun, 29 Mar 2015 12:37:34 -0600
To:	Sullivan Beck via RT <bug-Date-Manip [...] rt.cpan.org>
From:	Glenn Golden <gdg [...] zplane.com>

Sullivan Beck via RT <bug-Date-Manip@rt.cpan.org> [2015-03-27 13:45:37 -0400]: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=103142 > > > Date::Manip supports the following documented formats: > > mmmD > mmmDDYY > mmmDDYYYY > > so you can say Jun 05 or June 052015, both of which are clear. The question > is: what does Jun 2015' mean? The docs state that this is the equivalent > of June 20, 2015. >

Looking thru Date::Manip::Date.3pm, it does supply a semantic for the string "June2015", i.e. it is to be interpreted as mmmDDYY, hence as June 20, 2015. But I can't see anything in that man page that assigns any semantic at all to the string "June 2015". Is there some place elsewhere in the DM6 documentation that says that model date forms (such as "mmmDDYY") are permitted to be expressed with arbitrary whitespace between the components? Honestly, as a huge fan of D::M over the years, I think you're shooting yourself in the foot with this one. I do understand that you are hewing to the "goal of working with complete dates", but in this case it seems that the other (and original) goal of providing a super-general parser that eats almost anything and provides a perl-ish DWIM-like result gets thrown under the bus of common-sense interpretation. In a room full of 100 people, all shown the string "June 2026", how many are going to interpret that as the complete date June 20, 1926 [the result obtained by DM6] rather than as the partial date "sometime in June, 2026" [as obtained by DM5]? IMO, very few. Show quoted text

> > I could add another function (like ParseIncompleteDate) which supported a > bunch of incomplete date strings, but I'm not sure that this would really > clarify the situation. >

Agree that that would not be very clean. How about another approach: Introduce an option to parse() that provides a heuristic compromise for dates which are specifically of the form "mmm ABCD", where ABCD is a digit string and AB is specifically 19 or 20? $date->parse($string, ['pd_heuristic', ...]); Could you consider something like this? If so, I'd be happy to help in any way with it, perhaps testing or doc. I'd even be willing to take a shot at a source patch if you can at point me in the right direction in the code and provide a few initial hints. Not sure I'd get anywhere -- my familiarity with OO perl is minimal -- but could at least give it a go.

Sun Mar 29 14:57:29 2015 gdg [...] zplane.com - Correspondence added

Subject:	Re: [rt.cpan.org #103142] ParseDate() unexpected behavior with certain partial date constructs
Date:	Sun, 29 Mar 2015 12:57:13 -0600
To:	Sullivan Beck via RT <bug-Date-Manip [...] rt.cpan.org>
From:	Glenn Golden <gdg [...] zplane.com>

Btw, I meant to mention in the last post: This "mmm YYYY" is not a contrived example created to annoy you, although it probably seems that way. :) It's a real world situation taken from a financial trading application. Partial dates of that form are almost universally used to specify the expiration date of futures contracts, because the specific day of expiration varies from month to month, and nobody cares about the specific day anyway. Hence, only month and year are typically used to specify it. For example, traders routinely talk about contracts like "June 2015 WTI oil futures", which would typically be expressed to a program as something like '2015-06', or '2015.06', or 'June 2015'. DM6 behaves as desired on the first, but the second form is interpreted as a time during today, coming out as 20150329 20:15:03 and the third as we've been discussing. (Btw, just out of curiosity, in the above interpretation of 2015.06, where does the "03" on the seconds come from?)

Mon Apr 20 07:33:31 2015 sbeck [...] cpan.org - Correspondence added

I have added a new config variable (Format_MMMYYYY) which will allow this to be parsed. See the documentation for details. It'll be available in the next release (6.50).

Mon Apr 20 07:36:05 2015 sullybeck [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #103142] ParseDate() unexpected behavior with certain partial date constructs
Date:	Mon, 20 Apr 2015 07:35:51 -0400
To:	bug-Date-Manip [...] rt.cpan.org
From:	Sullivan Beck <sullybeck [...] gmail.com>

On 03/29/2015 02:57 PM, gdg@zplane.com via RT wrote: Show quoted text

> Queue: Date-Manip > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=103142 > > > Btw, I meant to mention in the last post: This "mmm YYYY" is not a contrived > example created to annoy you, although it probably seems that way. :) It's > a real world situation taken from a financial trading application. > > Partial dates of that form are almost universally used to specify the > expiration date of futures contracts, because the specific day of expiration > varies from month to month, and nobody cares about the specific day anyway. > Hence, only month and year are typically used to specify it. > > For example, traders routinely talk about contracts like "June 2015 WTI oil > futures", which would typically be expressed to a program as something like > '2015-06', or '2015.06', or 'June 2015'. DM6 behaves as desired on the first, > but the second form is interpreted as a time during today, coming out as > > 20150329 20:15:03 > > and the third as we've been discussing. > > (Btw, just out of curiosity, in the above interpretation of 2015.06, where > does the "03" on the seconds come from?) >

I got a 2nd request for this a couple of days ago (I suspect a conspiracy :-). In any case, after some consideration, I have added a new config variable Format_MMMYYYY which will allow parsing this truncated date format. Check out the Date::Manip::Config doc for detail. It'll be in the next release, or you can download a preview copy at: http://sullybeck.com/Date-Manip-6.50.tar.gz

Mon Apr 20 08:48:56 2015 gdg [...] zplane.com - Correspondence added

Subject:	Re: [rt.cpan.org #103142] ParseDate() unexpected behavior with certain partial date constructs
Date:	Mon, 20 Apr 2015 06:48:33 -0600
To:	Sullivan Beck via RT <bug-Date-Manip [...] rt.cpan.org>
From:	Glenn Golden <gdg [...] zplane.com>

Sullivan Beck via RT <bug-Date-Manip@rt.cpan.org> [2015-04-20 07:36:06 -0400]: Show quoted text

> > I got a 2nd request for this a couple of days ago (I suspect a > conspiracy :-). >

Ahhhh, Secret Agent 0042 finally came in from the cold... :) Show quoted text

> > In any case, after some consideration, I have added a new config > variable Format_MMMYYYY which will allow parsing this truncated date > format. Check out the Date::Manip::Config doc for detail. It'll be in > the next release, or you can download a preview copy at: > > http://sullybeck.com/Date-Manip-6.50.tar.gz >

I will check it out later this week. Thank you once again for your most excellent program and responsiveness on this issue! Now about the parsing efficiency for offset-only dates..... :) Just kidding, I'll stop haranguing you on this one for a while... Regards, Glenn