Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the DateTime CPAN distribution.

Report information
The Basics
Id: 96452
Status: resolved
Priority: 0/
Queue: DateTime

People
Owner: Nobody in particular
Requestors: snaury [...] gmail.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in:
  • 1.28
  • 1.29-TRIAL
  • 1.30-TRIAL
  • 1.31-TRIAL
  • 1.32
  • 1.33
  • 1.34
  • 1.35
  • 1.36
  • 1.37-TRIAL



Subject: DateTime->from_epoch loses precision when used with Time::HiRes
Date: Sat, 14 Jun 2014 21:06:14 +0400
To: bug-DateTime [...] rt.cpan.org
From: Alexey Borzenkov <snaury [...] gmail.com>
DateTime->from_epoch uses regexes to match floating point numbers and this causes it to lose a lot of precision (tested on 64-bit Ubuntu 14.04). For example: use DateTime; use Time::HiRes; foreach (1..10) { printf("%09d\n", DateTime->from_epoch(epoch => Time::HiRes::time)->nanosecond); } This code would show that only the first 5 numbers are non-zero. However, if the code is changed to: use DateTime; use Time::HiRes; foreach (1..10) { printf("%09d\n", DateTime->from_epoch(epoch => sprintf("%.9f", Time::HiRes::time))->nanosecond); } Then it will show the full nanosecond precision, as expected. I suspect that perl uses very little precision when stringifying floating point numbers (can't seem to find how it does it in the code), so by using regexes a lot of that precision is just lost. I think DateTime should only use regexes for the integer part, but to get to nanoseconds it should just substract the integer part from the given epoch, e.g. something like this: $args{nanosecond} = int(abs($p{epoch} - $int) * MAX_NANOSECONDS) if $dec;
That's not a DateTime issue. See https://metacpan.org/pod/Time::HiRes#time -> Note 2 for more information. You can see that perl -MTime::HiRes -E 'say Time::HiRes::time' returns only five digits. So this is a NOT_A_BUG
Created pull request with new paragraph in documenation: https://github.com/autarch/DateTime.pm/pull/6
On 2015-01-02 03:58:41, RENEEB wrote: Show quoted text
> That's not a DateTime issue. See > https://metacpan.org/pod/Time::HiRes#time -> Note 2 for more > information. > > You can see that > > perl -MTime::HiRes -E 'say Time::HiRes::time' > > returns only five digits. So this is a NOT_A_BUG
Well, I have to disagree. Time::HiRes creates an NV with full precision, and you can see the full precision if using sprintf or so: perl -MTime::HiRes -E 'say sprintf "%.9f", Time::HiRes::time' DateTime apparently turns this NV into a PV with perl's default format string, which keeps only 5 digits after the dot. I think Alexey's suggestion is quite reasonable.
On 2015-01-02 05:38:17, SREZIC wrote: Show quoted text
> On 2015-01-02 03:58:41, RENEEB wrote:
> > That's not a DateTime issue. See > > https://metacpan.org/pod/Time::HiRes#time -> Note 2 for more > > information. > > > > You can see that > > > > perl -MTime::HiRes -E 'say Time::HiRes::time' > > > > returns only five digits. So this is a NOT_A_BUG
> > Well, I have to disagree. Time::HiRes creates an NV with full > precision, and you can see the full precision if using sprintf or so: > > perl -MTime::HiRes -E 'say sprintf "%.9f", Time::HiRes::time' > > DateTime apparently turns this NV into a PV with perl's default format > string, which keeps only 5 digits after the dot. I think Alexey's > suggestion is quite reasonable.
BTW it's possible to reproduce the issue without Time::HiRes at all: $ perl -MDateTime -e 'warn DateTime->from_epoch(epoch => 1420195665.123456)->nanosecond' 123460000 at -e line 1. Again only 5 significant digits here. A workaround is to use a string instead: $ perl -MDateTime -e 'warn DateTime->from_epoch(epoch => "1420195665.123456")->nanosecond' 123456000 at -e line 1.
It's not easy to fix it in an other way. If you just do "sprintf "%.9f"" or the like, you'll break other stuff. See t/13stftime.t ([1]). Those tests will fail then. [1] { # Internally this becomes 119999885 nanoseconds (floating point math is awesome) my $epoch = 1297777805.12; my $dt = DateTime->from_epoch( epoch => $epoch ); my @vals = ( 1, 12, 120, 1200, 12000, 120000, 1200000, 12000000, 120000000, 1200000000, ); my $x = 1; for my $val (@vals) { my $spec = '%' . $x++ . 'N'; is( $dt->strftime($spec), $val, "strftime($spec) for $epoch == $val" ); } }
On 2015-01-02 07:06:13, RENEEB wrote: Show quoted text
> It's not easy to fix it in an other way. If you just do "sprintf > "%.9f"" or the like, you'll break other stuff. See t/13stftime.t > ([1]). Those tests will fail then. > > > [1] > { > # Internally this becomes 119999885 nanoseconds (floating point > math is awesome) > my $epoch = 1297777805.12; > my $dt = DateTime->from_epoch( epoch => $epoch ); > > my @vals = ( > 1, > 12, > 120, > 1200, > 12000, > 120000, > 1200000, > 12000000, > 120000000, > 1200000000, > ); > > my $x = 1; > for my $val (@vals) { > my $spec = '%' . $x++ . 'N'; > is( > $dt->strftime($spec), $val, > "strftime($spec) for $epoch == $val" > ); > } > }
This is shocking, but why should DateTime be better than perl (or the machine's floating point representation) here? $ perl -E 'say sprintf("%.9f", 1297777805.12)' 1297777805.119999886
On 2015-01-02 10:08:35, SREZIC wrote: Show quoted text
> On 2015-01-02 07:06:13, RENEEB wrote:
> > It's not easy to fix it in an other way. If you just do "sprintf > > "%.9f"" or the like, you'll break other stuff. See t/13stftime.t > > ([1]). Those tests will fail then. > > > > > > [1] > > { > > # Internally this becomes 119999885 nanoseconds (floating point > > math is awesome) > > my $epoch = 1297777805.12; > > my $dt = DateTime->from_epoch( epoch => $epoch ); > > > > my @vals = ( > > 1, > > 12, > > 120, > > 1200, > > 12000, > > 120000, > > 1200000, > > 12000000, > > 120000000, > > 1200000000, > > ); > > > > my $x = 1; > > for my $val (@vals) { > > my $spec = '%' . $x++ . 'N'; > > is( > > $dt->strftime($spec), $val, > > "strftime($spec) for $epoch == $val" > > ); > > } > > }
> > This is shocking, but why should DateTime be better than perl (or the > machine's floating point representation) here? > > $ perl -E 'say sprintf("%.9f", 1297777805.12)' > 1297777805.119999886
More thoughts: * Don't use floats at all, stay integer: perl5.20.1 -MTime::HiRes=gettimeofday -MDateTime -e '$t0 = [gettimeofday]; $dt = DateTime->from_epoch(epoch => $t0->[0]); $dt->set_nanosecond($t0->[1]*1000); printf "%09d\n", $dt->nanosecond' will give the expected precision (6 digits, as gettimeofday has only microsecond resolution) (note, Time::HiRes::time() is using gettimeofday() internally). A convenience method in DateTime would be nice, though, maybe: DateTime->from_epoch(gettimeofday => Time::HiRes::gettimeofday()) * A perl compiled with -Duselongdouble does not have the floating point problems, at least for the test value in the mentioned DateTime test: $ perl5.20.1D -E 'say sprintf("%.9f", 1297777805.12)' 1297777805.120000000 Who knows, maybe -Duselongdouble will be standard in the future, like 64bit integers are standard now?
I have submitted a PR which changes the behavior in ->from_epoch(). <https://github.com/autarch/DateTime.pm/pull/11> -- chansen
I merged chansen's PR a while back (in 1.28)