Subject: | Precision does not behave as documented (doc confusion or bug) |
Date: | Tue, 8 Mar 2016 14:55:52 -0800 |
To: | bug-Math-BigFloat [...] rt.cpan.org |
From: | Jim Avera <jim.avera [...] gmail.com> |
Hi, The pod for Math::BigFLoat has a section "ACCURACY and PRECISION".
It contains a table giving examples of different Precision values and
how they affect rounded results.
By my reading, it shows completely wrong values, essentially opposite to
actual behavior.
Of course, I could be interpreting the table backwards somehow; if so,
please correct me and
consider adding something to the docs so others won't have the same
confusion as me.
Here's the problem I see: The pod says this:
Initial value P A Result String
------------------------------------------------------------
1234.01 -3 1000 1000
1234 -2 1200 1200
1234.5 -1 1230 1230
1234.001 1 1234 1234.0
1234.01 0 1234 1234
1234.01 2 1234.01 1234.01
1234.01 5 1234.01 1234.01000
But actual behavior is this:
Initial val P String
1234.01 -3 1234.010
1234 -2 1234.00
1234.5 -1 1234.5
1234.001 1 1234
1234.01 2 1230
1234.01 5 0
And here is the program used to generate the "actual" results:
#!/usr/bin/perl
use strict; use warnings;
use Math::BigFloat;
printf "%-11s %2s %s\n", "Initial val", "P", "String";
foreach(
[ "1234.01", -3 ], [ "1234", -2 ], [ "1234.5", -1 ],
[ "1234.001", 1 ], [ "1234.01", 2 ], [ "1234.01", 5 ],
) {
my ($init_str, $P) = @$_;
my $x = Math::BigFloat->new($init_str);
$x->round(undef,$P);
printf "%-11s %2d %s\n", $init_str, $P, scalar($x);
}