Bug #123613 for List-MoreUtils-XS: Fails on perls >= 5.27.4 built with -DDEBUGGING

Sun Nov 12 17:10:25 2017 ANDK [...] cpan.org - Ticket created

Subject:

Fails on perls >= 5.27.4 built with -DDEBUGGING

As per subject. Diagnostics: panic: XSUB List::MoreUtils::XS::arrayify (XS.c) failed to extend arg stack: base=55a8a93d0a20, sp=55a8a93d0ab0, hwm=55a8a93d0a80 t/xs/arrayify.t ............. Dubious, test returned 255 (wstat 65280, 0xff00) No subtests run This is taken from this report: http://www.cpantesters.org/cpan/report/01561956-ba09-11e7-af6a-d958478e6c56 Perl was bleadperl v5.27.5-87-gae0af6eca6 in this report, but it's just a random pick. My smokers produced a couple of other FAIL reports with perls between 5.27.4 and current bleadperl (last one was v5.27.5-120-ga17768d7c7), but only when -DDEBUGGING was on.

Mon Nov 13 23:56:16 2017 ANDK [...] cpan.org - Correspondence added

The panic in the above cited report was introduced in v5.27.3-29-gcfbdacd. So I tried some earlier bleadperls and finally went back to v5.27.1-66-g87058c31e9 which introduced a similar check and indeed I could also produce the other panic on that perl. The diagnostics there in a similar tune: panic: previous op failed to extend arg stack: base=5562f7a0cb50, sp=5562f7a0cbe0, hwm=5562f7a0cbb0 The full report has been sent to cpantesters and will be available from there once it has gone through their goods receipt.

Tue Nov 14 12:04:34 2017 REHSACK [...] cpan.org - Correspondence added

I've realized the report - but unfortunately I wont have time before Christmas to have a deeper look. If you have any updates, please inform me via this ticket. Otherwise I'll dig into it using the most recent -blead when it find a tuit.

Tue Nov 14 12:04:35 2017 The RT System itself - Status changed from 'new' to 'open'

Wed Nov 15 16:35:30 2017 ANDK [...] cpan.org - Correspondence added

Reading material in https://rt.perl.org/Public/Bug/Display.html?id=131938

Fri Nov 17 03:29:11 2017 ANDK [...] cpan.org - Correspondence added

RT-Send-CC:

TONYC [...] cpan.org

Attached patch seems to fix the issue, at least all tests pass both with recent bleadperls built with -DDEBUGGING and older normal perls. Tony, would you mind having a look at the fix whether it looks valid? Thanks!

Subject:

List-MoreUtils-XS-0.426-123613.patch

--- XS.xs~ 2017-09-27 14:10:33.000000000 +0000 +++ XS.xs 2017-11-15 22:21:59.847008133 +0000 @@ -1613,7 +1613,7 @@ arrayify(...) CODE: { - I32 i; + I32 i,j; AV *rc = newAV(); AV *args = av_make(items, &PL_stack_base[ax]); sv_2mortal(newRV_noinc((SV *)rc)); @@ -1621,7 +1621,9 @@ LMUav2flat(aTHX_ rc, args); - for(i = AvFILLp(rc); i >= 0; --i) + j = AvFILLp(rc); + EXTEND(SP,j); + for(i = j; i >= 0; --i) { ST(i) = sv_2mortal(AvARRAY(rc)[i]); AvARRAY(rc)[i] = NULL;

Fri Nov 17 03:50:56 2017 tony [...] develop-help.com - Correspondence added

CC:	TONYC [...] cpan.org
Subject:	Re: [rt.cpan.org #123613] Fails on perls >= 5.27.4 built with -DDEBUGGING
Date:	Fri, 17 Nov 2017 19:50:22 +1100
To:	Andreas Koenig via RT <bug-List-MoreUtils-XS [...] rt.cpan.org>
From:	Tony Cook <tony [...] develop-help.com>

On Fri, Nov 17, 2017 at 03:29:11AM -0500, Andreas Koenig via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=123613 > > > Attached patch seems to fix the issue, at least all tests pass both with recent bleadperls built with -DDEBUGGING and older normal perls. > > Tony, would you mind having a look at the fix whether it looks valid? > > Thanks!

Show quoted text

> --- XS.xs~ 2017-09-27 14:10:33.000000000 +0000 > +++ XS.xs 2017-11-15 22:21:59.847008133 +0000 > @@ -1613,7 +1613,7 @@ > arrayify(...) > CODE: > { > - I32 i; > + I32 i,j; > AV *rc = newAV(); > AV *args = av_make(items, &PL_stack_base[ax]); > sv_2mortal(newRV_noinc((SV *)rc)); > @@ -1621,7 +1621,9 @@ > > LMUav2flat(aTHX_ rc, args); > > - for(i = AvFILLp(rc); i >= 0; --i) > + j = AvFILLp(rc); > + EXTEND(SP,j);

j is the index of the last element, but the following code pushes the "number of elements" items, so this should be: EXTEND(SP,j+1); Show quoted text

> + for(i = j; i >= 0; --i) > { > ST(i) = sv_2mortal(AvARRAY(rc)[i]); > AvARRAY(rc)[i] = NULL;

Tony

Fri Nov 17 10:11:54 2017 andreas.koenig.7os6VVqR [...] franz.ak.mind.de - Correspondence added

CC:	ANDK [...] cpan.org, TONYC [...] cpan.org
Subject:	Re: [rt.cpan.org #123613] Fails on perls >= 5.27.4 built with -DDEBUGGING
Date:	Fri, 17 Nov 2017 16:11:32 +0100
To:	"tony\ [...] develop-help.com via RT" <bug-List-MoreUtils-XS [...] rt.cpan.org>
From:	Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>

Show quoted text

> EXTEND(SP,j+1);

Doh! Thanks for spotting that, Tony! -- andreas

Tue Dec 12 14:06:07 2017 REHSACK [...] cpan.org - Correspondence added

Thanks for bringing this up. Indeed, the stack growing was missing. A minimalist patch would look like: @@ -1621,6 +1621,7 @@ CODE: LMUav2flat(aTHX_ rc, args); + EXTEND(SP, AvFILLp(rc)+1); for(i = AvFILLp(rc); i >= 0; --i) { ST(i) = sv_2mortal(AvARRAY(rc)[i]); For documentation purposes we deciced extract the common variable and release 0.427_001 for testing.

Tue Dec 12 14:06:33 2017 REHSACK [...] cpan.org - Status changed from 'open' to 'patched'

Tue Dec 12 18:34:06 2017 ether [...] cpan.org - Correspondence added

I noticed that the 'Changes' file references the wrong ticket -- it says 123631 not 12613.

Tue Jan 02 05:00:58 2018 REHSACK [...] cpan.org - Status changed from 'patched' to 'resolved'

Tue Jan 02 05:00:59 2018 REHSACK [...] cpan.org - Fixed in 0.428 added