Skip Menu |

This queue is for tickets about the Math-BigInt-GMP CPAN distribution.

Report information
The Basics
Id: 103517
Status: resolved
Priority: 0/
Queue: Math-BigInt-GMP

People
Owner: Nobody in particular
Requestors: CHORNY [...] cpan.org
Cc: dom [...] cpan.org
gregoa [...] cpan.org
AdminCc:

Bug Information
Severity: Normal
Broken in: 1.38
Fixed in: 1.51



Subject: number changes sign after new
use 5.012; use warnings; use Math::BigInt 'lib' => 'GMP'; #,Pari,FastCalc my $secs = 2_148_137_600; say "\$secs =$secs"; my $secs1 = Math::BigInt->new($secs); say "\$secs1=$secs1"; $secs = 2148137600 $secs1= -214682969 latest Math::BigInt and Math::BigInt::GMP 32-bit Windows XP, 32-bit Debian 7 -- Alexandr Ciornii, http://chorny.net
I have compiled a 32 bit version of Perl 5.20.2 just for this bug report, but I am unable to reproduce your results. Do you get the same results if you are not specifying the library explicitly? What output do you get from the following two commands? perl -MMath::BigInt -wle 'print Math::BigInt -> new(2_148_137_600)' perl -MMath::BigInt -wle 'print Math::BigInt -> VERSION'
On Thu Apr 16 10:45:31 2015, pjacklam wrote: Show quoted text
> I have compiled a 32 bit version of Perl 5.20.2 just for this bug > report, but I am unable to reproduce your results. Do you get the same > results if you are not specifying the library explicitly? > > What output do you get from the following two commands? > > perl -MMath::BigInt -wle 'print Math::BigInt -> new(2_148_137_600)'
2148137600 Show quoted text
> > perl -MMath::BigInt -wle 'print Math::BigInt -> VERSION'
1.9993 Result is correct because GMP is not used. $ perl -MMath::BigInt -wle 'print Math::BigInt->config()->{lib};' Math::BigInt::Calc -- Alexandr Ciornii, http://chorny.net
Thanks. I missed the point that it *only* happens with the GMP backend. I was able to reproduce your result: $ /opt/perl5/cygwin_nt-6.1-wow/1.7.35/i686/bin/5.20.2/cygwin/perl -MMath::BigInt=lib,GMP -wle 'print Math::BigInt -> new(2_148_137_600)' -214682969 However, I am not sure if this really is an error. I suspect the negative output occurs because your input value is larger than 2_147_483_647, which is the largest value that can be represented by a 32 bit signed integer. See what happens when you cross this boundary: $ perl -MMath::BigInt=lib,GMP -wle 'print Math::BigInt -> new(2_147_483_647)' 2147483647 $ perl -MMath::BigInt=lib,GMP -wle 'print Math::BigInt -> new(2_147_483_648)' -214748364 The issue at hand is related to the following section in the documentation for Math::BigInt: Input values to these routines may be any string, that looks like a number and results in an integer, including hexadecimal and binary numbers. Scalars holding numbers may also be passed, but note that non-integer numbers may already have lost precision due to the conversion to float. Quote your input if you want BigInt to see all the digits: $x = Math::BigInt->new(12345678890123456789); # bad $x = Math::BigInt->new('12345678901234567890'); # good The wording could be improved, but the point is that input to Math::BigInt->new() should be quoted, at least when the input has a large number of digits. When quoting your input value, the output is correct: $ perl -Ilib -MMath::BigInt=lib,GMP -wle 'print Math::BigInt -> new("2_148_137_600")' 2148137600
Duplicate of 71548? This is a real PITA bug, as it forces quoting of every input or adding a bunch of hacky code to check for things like - is the input a class (perhaps already BigInt)? Alternately pay the penalty for stringify / unstringify. - is the input large enough to hit this? - is this the GMP backend that acts differently than every other backend? As seen in that ticket, you can easily cause perl to die with floating point exceptions because of this. But only with this backend. What I'm seeing is that SvUOK(x) returns true for large integers (positive with high bit set). Hence we're doing exactly the opposite of what we should be doing in GMP.xs. We're taking a number Perl has told us "this is a number that fits in 32-/64- bit, is positive, and has the high bit set" and we intentionally reverse its sign. This can't be right. Arguably we should get clever with SvFLAGS to only use the string set when it is necessary.
Subject: Re: [rt.cpan.org #103517] number changes sign after new
Date: Mon, 8 Jun 2015 13:32:25 -0400
To: bug-Math-BigInt-GMP [...] rt.cpan.org
From: "Tels" <nospam-abuse [...] bloodgate.com>
On Sun, June 7, 2015 6:17 pm, Dana Jacobsen via RT wrote: Show quoted text
> Queue: Math-BigInt-GMP > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=103517 > > > Duplicate of 71548? > > This is a real PITA bug, as it forces quoting of every input or adding a > bunch of hacky code to check for things like > - is the input a class (perhaps already BigInt)? Alternately pay the > penalty for stringify / unstringify. > - is the input large enough to hit this? > - is this the GMP backend that acts differently than every other backend? > > As seen in that ticket, you can easily cause perl to die with floating > point exceptions because of this. But only with this backend. > > What I'm seeing is that SvUOK(x) returns true for large integers (positive > with high bit set). Hence we're doing exactly the opposite of what we > should be doing in GMP.xs. We're taking a number Perl has told us "this > is a number that fits in 32-/64- bit, is positive, and has the high bit > set" and we intentionally reverse its sign. This can't be right. > > Arguably we should get clever with SvFLAGS to only use the string set when > it is necessary.
FWIIW, I agree. Quoting the input to BigInt should not be necessary if the input fits into an integer - the documentation (even if not that clearly worded) only covers cases where the internal representation of Perl's INT cannot hold the given value and thus the bits are lost before they reach BigInt/BigFloat's new(). In this case the bits reach the library, but are lost in (or on the way to) the backend. Which shouldn't happen or be visible to the user. Best regards, Tels -- http://bloodgate.com/galleries/
I don't know XS well enough to fix this, but if someone else provides a patch, I'd be happy to include it in a (not too distant) future release.
This is a patch version of what I indicated in May 2013: --- GMP.xs.old 2015-12-01 16:10:02.000000000 -0800 +++ GMP.xs 2015-12-01 19:04:39.000000000 -0800 @@ -169,7 +169,7 @@ /* using the IV directly is a bit faster */ if (SvUOK(x)) { - mpz_init_set_si(*RETVAL, (UV)SvUV(x)); + mpz_init_set_ui(*RETVAL, (UV)SvUV(x)); } else { @@ -190,7 +190,7 @@ CODE: mpz = malloc (sizeof(mpz_t)); if (SvUOK(x)) { - mpz_init_set_si(*mpz, (UV)SvUV(x)); + mpz_init_set_ui(*mpz, (UV)SvUV(x)); } else { mpz_init_set_str(*mpz, SvPV_nolen(x), 10); Math::BigInt's new() strips the sign off the input, so anything coming to _new is unsigned. The defect is that _new is coercing the UV into a signed long. Regarding performance, unfortunately since Math::BigInt removes the sign by using s/// that means any signed input comes in looking like a string so goes through the slower string path. Probably not a huge problem since most inputs are likely unsigned. The following is a bit more efficient, as it results in many more integer inputs going through the faster path: --- GMP.xs.old 2015-12-01 16:10:02.000000000 -0800 +++ GMP.xs 2015-12-01 19:11:28.000000000 -0800 @@ -167,9 +167,9 @@ CODE: NEW_GMP_MPZ_T; /* using the IV directly is a bit faster */ - if (SvUOK(x)) + if (SvUOK(x) || SvIOK(x)) { - mpz_init_set_si(*RETVAL, (UV)SvUV(x)); + mpz_init_set_ui(*RETVAL, (UV)SvUV(x)); } else { @@ -189,8 +189,8 @@ mpz_t *mpz; CODE: mpz = malloc (sizeof(mpz_t)); - if (SvUOK(x)) { - mpz_init_set_si(*mpz, (UV)SvUV(x)); + if (SvUOK(x) || SvIOK(x)) { + mpz_init_set_ui(*mpz, (UV)SvUV(x)); } else { mpz_init_set_str(*mpz, SvPV_nolen(x), 10); All current tests pass with this patch. It is, of course, possible that the test suite isn't complete. I'm also attaching a test file. When run with Calc or Pari back end, it passes all tests in both 32-bit and 64-bit Perl. With the current GMP back end, it fails 4 tests and hangs or dies on a fifth without the patch. This test exercises code I run every day, but have to work around by forcing all arguments to new() into strings, just in case the GMP back end is being used and the input falls into the bad range. With either patch above, this passes all tests just like the other back ends. I don't think this test file is ready to get dropped into the test suite, but portions of it may be appropriate to add. I think the tests for: -(~0 >> 1 + 1) ~0 >> 1 ~0 >> 1 + 1 ~0 - 1 ~0 would be good to see.
Subject: bigint-new.t
#!/usr/bin/env perl use strict; use warnings; use Test::More; use Math::BigInt lib=>"GMP"; my $use64 = ~0 > 4294967295; my $broken64 = (18446744073709550592 == ~0); die "Your 64-bit system is broken. Upgrade from 5.6 for this test." if $broken64; plan tests => 4*2 + 2*1 + 1 + $use64; my $maxs = ~0 >> 1; for my $n ($maxs - 2, $maxs - 1, $maxs, $maxs+1) { is( Math::BigInt->new($n), $n, "new $n" ); is( Math::BigInt->new(-$n), -$n, "new -$n" ); } for my $n (~0 - 1, ~0) { is( Math::BigInt->new($n), $n, "new $n" ); } # bacmp makes a new variable. This will test if it is screwing up the sign. is( Math::BigInt->new(10)->bacmp(~0), -1, "10 should be less than maxint" ); if ($use64) { diag "The following test may hang or cause an exception if incorrect"; is( Math::BigInt->new("14")->bmodpow(9506577562092332135, "29544731879021791655795710"), "19946192910281559497582964", "big modpow" ); }
Thanks! I have uploaded Math-BigInt-GMP-1.46 to PAUSE based on the above fix.
RT-Send-CC: DANAJ [...] cpan.org
On Tue Dec 01 22:22:12 2015, DANAJ wrote: Show quoted text
> This is a patch version of what I indicated in May 2013: (...)
Your patch causes problems when Perl is compiled with 64-bit support, but the underlying OS is 32-bit. Alas, I didn't test your patch with that configuration before I released a new version of Math-BigInt-GMP. Here you can see the failing tests: http://matrix.cpantesters.org/?dist=Math-BigInt-GMP%201.46 And here is a case from the old test suite which fails with the latest release (the one with your patch added): $ perl -MMath::BigInt::GMP -wle '$x = Math::BigInt::GMP->_new(1050000000000000); print Math::BigInt::GMP::_str("Math::BigInt::GMP", $x)' 755212288 This case worked fine without your patch. I would appreciate any suggestions about how to fix this.
On Fri Dec 04 08:14:57 2015, pjacklam wrote: Show quoted text
> Your patch causes problems when Perl is compiled with 64-bit support, > but the underlying OS is 32-bit. Alas, I didn't test your patch with > that configuration before I released a new version of Math-BigInt-GMP.
Doh! At least it's caught by tests. The problem stems from a few sources: 1) GMP's API uses signed or unsigned long, while Perl's UV may be larger than that. This makes blind use of the _ui and _si functions problematic. A brief look at the rest of the .xs file seems to say this won't typically be a problem elsewhere (arguments to shift amounts and non-modular power). 2) SvUOK was already broken for 2^63+, we just didn't test for it. That was this RT. 3) The optimization for SvIOK made machines with smaller longs break for 2^32+. We do test for that. But just removing that optimization means we're still broken for #1. So either: if (sizeof(UV) <= sizeof(unsigned long) && (SvUOK(x) || SvIOK(x))) { mpz_init_set_ui(*RETVAL, (unsigned long)SvUV(x)); } replacing the original lines in both _new and _new_attach. This says that we will go to strings all the time if UV is bigger. We could do a little better but it's getting messy: if ((SvUOK(x) || SvIOK(x)) && (sizeof(UV) <= sizeof(unsigned long) || SvUV(x) == (unsigned long)SvUV(x))) { mpz_init_set_ui(*RETVAL, (unsigned long)SvUV(x)); } which is checking that the input is the same as a UV and unsigned long. This could be optimized to just call SvUV once, if this was the path to go down. Show quoted text
> And here is a case from the old test suite which fails with the latest > release (the one with your patch added): > > $ perl -MMath::BigInt::GMP -wle '$x = Math::BigInt::GMP-
> >_new(1050000000000000); print
> Math::BigInt::GMP::_str("Math::BigInt::GMP", $x)' > 755212288
Verified on Cygwin and Debian Athlon II, both the 4 byte longs and 8 byte UVs. Either of the changes above fixes this. Feel free to email me directly as well, and I'd welcome input from anyone else on this.
I've converted Dana's (second) suggestion into a patch now and built the package on Debian unstable in an amd64 and an i386 environment, and it builds and passes its test suite in both cases. I'm attaching the patch. Any comments (or preferrably a new release of Math-BigInt-GMP) would be very welcome so that we can get this fixed. Cheers, gregor, Debian Perl Group
Subject: 32bit.patch
Description: fix breakage on 32 bit platforms where perl is compiled with 64 bit support Origin: CPAN RT#103517 Bug: https://rt.cpan.org/Public/Bug/Display.html?id=103517 Bug-Debian: https://bugs.debian.org/807492 Author: Dana A Jacobsen <danaj@cpan.org> Last-Update: 2015-12-13 --- a/GMP.xs +++ b/GMP.xs @@ -167,9 +167,9 @@ CODE: NEW_GMP_MPZ_T; /* using the IV directly is a bit faster */ - if (SvUOK(x) || SvIOK(x)) + if ((SvUOK(x) || SvIOK(x)) && (sizeof(UV) <= sizeof(unsigned long) || SvUV(x) == (unsigned long)SvUV(x))) { - mpz_init_set_ui(*RETVAL, (UV)SvUV(x)); + mpz_init_set_ui(*RETVAL, (unsigned long)SvUV(x)); } else { @@ -189,8 +189,8 @@ mpz_t *mpz; CODE: mpz = malloc (sizeof(mpz_t)); - if (SvUOK(x) || SvIOK(x)) { - mpz_init_set_ui(*mpz, (UV)SvUV(x)); + if ((SvUOK(x) || SvIOK(x)) && (sizeof(UV) <= sizeof(unsigned long) || SvUV(x) == (unsigned long)SvUV(x))) { + mpz_init_set_ui(*mpz, (unsigned long)SvUV(x)); } else { mpz_init_set_str(*mpz, SvPV_nolen(x), 10);

Message body is not shown because it is too large.

Subject: Re: [rt.cpan.org #103517] number changes sign after new
Date: Mon, 14 Dec 2015 21:52:22 +0100
To: Peter John Acklam via RT <bug-Math-BigInt-GMP [...] rt.cpan.org>
From: gregor herrmann <gregoa [...] debian.org>
On Mon, 14 Dec 2015 14:37:02 -0500, Peter John Acklam via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=103517 > > I have released Math-BigInt-GMP-1.47, which seems to fix the issue in this ticket.
Thank you, uploaded to debian/unstable. We can follow the results of our buildds at https://buildd.debian.org/status/package.php?p=libmath-bigint-gmp-perl Cheers, gregor -- .''`. Homepage: http://info.comodo.priv.at/ - OpenPGP key 0xBB3A68018649AA06 : :' : Debian GNU/Linux user, admin, and developer - https://www.debian.org/ `. `' Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe `- NP: Bruce Springsteen: Highway Patrolman
Download signature.asc
application/pgp-signature 949b

Message body not shown because it is not plain text.

This ticket can be closed. 1.51 contains the patch already and smokes are clear. -- Reini Urban