Skip Menu |

This queue is for tickets about the Math-Clipper CPAN distribution.

Report information
The Basics
Id: 129486
Status: resolved
Priority: 0/
Queue: Math-Clipper

People
Owner: Nobody in particular
Requestors: khw [...] cpan.org
Cc: jkeenan [...] cpan.org
smueller [...] cpan.org
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: 1.29



CC: smueller [...] cpan.org
Subject: blead and perl 5.30 cause the test suite to fail on Free BSD and Open BSD
A work around was added to Math-Clipper in May 2018 for a bug in the pow() function on Free BSD machines. It turns out that Open BSD is also affected. The work around involves these two lines from Clipper.pm # brings behavior of FreeBSD + clang systems into harmony with others $scale_vector[$ci] = 0.0 + sprintf("%.0f",$scale_vector[$ci]); (If the comment had been clearer, it would have saved us some debugging time, though maybe the author didn't fully understand the situation) It turns out that the earlier exponentiation 10**(-$max_exp + ($intspecs{$opts{bits}}->{maxdigits} - 1)); returns a slightly too large value when the exponent there evaluates to 29. (This is regular double precision; I havent tested with long doubles) And it turns out that this statement turns, underneath the hood, into a call of the library routine pow(10,29), and that is buggy on these platforms. The 'harmonizing' line executes a sprintf which stringifies this wrong value, and adding it to zero causes perl to parse that string, and converted the wrong value into the right one. That is, it did so, until we fixed it to parse the string correctly, retaining the wrong value, causing the suite to fail. I have no idea what the extent of the pow bug is. It could be just these two values, or it could be lots of inputs cause failures. So what to do about this. We have tested our fix of string parsing with many different values, and it works better than what was there before, which fails miserably on many. Hence, we aren't going to revert our change. It likely fixes bugs with other inputs that your suite doesn't happen to check. perl is not going to furnish a pow() function to override broken ones on those few platforms that have failures. What could you do? For one, the call to _floor is asymmetric with regards to positive vs negative values. I think it should truncate towards zero. As it is, -2.000000000...0000001 gets changed to -3. I think that should be changed. Either a different function, such as rint() should be used, or perhaps easier is to take the floor of the absolute value, and then change the sign back to negative if needed. Doing that causes the test suite to pass, and the workaround there now wouldn't be necessary in any perl version. A less enticing option would be to replace the exponentiation with a loop that multiplies 10 by itself n times. But Math::Clipper shouldn't have to write its own pow() equivalent for a bug on one platform. (Although the 'harmonizing' line is kinda doing that)
I forgot to include a link to the ticket I filed on Free BSD https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237800
On Wed May 08 11:37:06 2019, khw wrote: Show quoted text
> I forgot to include a link to the ticket I filed on Free BSD > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237800
And it looks like they are not going to fix this, as the results are within their view of tolerances. Tony Cook looked at the C standard and did not find any language as to acceptable precision in the return of these sorts of functions.
I'll take a look. Thanks for the analysis.
Math::Clipper 1.28 up on CPAN now hopefully fixes this. Tests pass on FreeBSD 11.2, perl 5.29.10 and perl-blead. I took the "less enticing option" and used a loop to do the exponentiation, because it's dead simple, and because I don't want to explore (again) new mysteries that might come up with floor() here. I think the loop has the advantage that if the scale factor calculated is an integer and fits within an IV, it should come out as a scalar holding an IV, since all the numbers in the calculation probably start out as IVs. So most scale factor calculations > 1 become immune to float imprecision, while the same loop handles floats for scales < 1.
On Sat May 11 20:19:45 2019, SHELDRAKE wrote: Show quoted text
> Math::Clipper 1.28 up on CPAN now hopefully fixes this. > > Tests pass on FreeBSD 11.2, perl 5.29.10 and perl-blead. >
$ ./bin/perl -Ilib -v | head -2 | tail -1; uname -mrs; ./bin/perl -MMath::Clipper -E 'say q|hello world|;' This is perl 5, version 30, subversion 0 (v5.30.0-RC1-3-g12e248a19e) built for amd64-freebsd-thread-multi FreeBSD 11.2-STABLE amd64 hello world Thank you very much. Jim Keenan
On Sat May 11 20:43:39 2019, JKEENAN wrote: Show quoted text
> On Sat May 11 20:19:45 2019, SHELDRAKE wrote:
> > Math::Clipper 1.28 up on CPAN now hopefully fixes this. > > > > Tests pass on FreeBSD 11.2, perl 5.29.10 and perl-blead. > >
> > $ ./bin/perl -Ilib -v | head -2 | tail -1; uname -mrs; ./bin/perl > -MMath::Clipper -E 'say q|hello world|;' > This is perl 5, version 30, subversion 0 (v5.30.0-RC1-3-g12e248a19e) > built for amd64-freebsd-thread-multi > FreeBSD 11.2-STABLE amd64 > hello world > > Thank you very much. > Jim Keenan
I have also been able to install Math::Clipper on OpenBSD-6.4 threaded.
CC: jkeenan [...] cpan.org, smueller [...] cpan.org
Subject: Re: [rt.cpan.org #129486] blead and perl 5.30 cause the test suite to fail on Free BSD and Open BSD
Date: Wed, 22 May 2019 13:02:47 -0600
To: bug-Math-Clipper [...] rt.cpan.org
From: Karl Williamson <khw [...] cpan.org>
On 5/11/19 6:19 PM, Michael E. Sheldrake via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=129486 > > > Math::Clipper 1.28 up on CPAN now hopefully fixes this. > > Tests pass on FreeBSD 11.2, perl 5.29.10 and perl-blead. > > I took the "less enticing option" and used a loop to do the exponentiation, because it's dead simple, and because I don't want to explore (again) new mysteries that might come up with floor() here. > > I think the loop has the advantage that if the scale factor calculated is an integer and fits within an IV, it should come out as a scalar holding an IV, since all the numbers in the calculation probably start out as IVs. So most scale factor calculations > 1 become immune to float imprecision, while the same loop handles floats for scales < 1. > >
Something I forgot to say was that several people questioned that the test was too restrictive in expecting that the final digit will be exact, and that off-by-one errors should be expected. Several referred to this paper: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html Indeed, someone wrote a program that did a similar loop, and got errors in the final digit, depending on the exact nature of the loop. Indeed, the way I would have written it had that error. And that way would be to take the floor of the square root of the exponent, and then do x = x*x that many times. Then take the square root of the undone part, and using the original value repeat the initial process and then multiply those results, and repeat until the whole exponent is gotten. This minimizes multiplications. Every operation introduces some error. But, on this person's platform that led to an off-by-one error in the final digit. Other schemes that used more multiplications didn't have this error. The bottom line is that its complicated
Thanks for the additional notes. Will take those into account next time this needs to be reconsidered. Closing this because the fixes are testing well, and it should be more sound than it was. I do expect to be able to round-trip coordinate data in and out of Clipper's C structures without the process changing last digits. This really is important when you do multiple Clipper operations, making several round trips. I also wanted to be really flexible with what kind of numerical data the integerize function would accept - a mix of floats, integers and even numbers-as-strings. If it weren't for that approach - if we could assume that all the Perl scalars coming in were doubles, or all were integers - the whole integerize approach could be more streamlined. So I agree, it's complicated - for the reasons you pointed to and more. Thanks to all for all the investigations that went on around this.