Skip Menu |

This queue is for tickets about the Digest-SHA CPAN distribution.

Report information
The Basics
Id: 86295
Status: resolved
Worked: 2.5 hours (150 min)
Priority: 0/
Queue: Digest-SHA
People
Owner: mshelor [...] cpan.org
Requestors: dom [...] cpan.org
Cc:
AdminCc:
Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: 5.85

History Show all quoted text
  Thu Jun 20 18:13:28 2013 dom [...] cpan.org - Ticket created
Subject: Another double free crash
A similar crash to RT#82655 was observed in <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=711206>. Unfortunately there is no minimal test case for this yet, but since it is so similar to the existing bug, I thought it might be worth forwarding it here in any case. Here is an example crash with 5.84: dom@perltest2:~/vanityhash-1.1$ ./vanityhash -b 32 -w 6 -d sha1 08b5124c447f </dev/null Reading input data and adding to digest...done. Original data: 0 bytes, SHA1 da39a3ee5e6b4b0d3255bfef95601890afd80709 Searching for 08b5124c447f at position 0 in a 32-bit space. Spawning 6 workers... done. ^CUser interrupt, cleaning up. *** Error in `/usr/bin/perl': double free or corruption (!prev): 0x09bdf828 *** ======= Backtrace: ========= /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x75e42)[0xb7630e42] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x76b80)[0xb7631b80] /usr/lib/perl5/auto/Digest/SHA/SHA.so(+0x637e)[0xb734737e] /usr/lib/perl5/auto/Digest/SHA/SHA.so(+0x6bd0)[0xb7347bd0] /usr/bin/perl(Perl_pp_entersub+0x5eb)[0x80f6e8b] /usr/bin/perl(Perl_runops_standard+0x18)[0x80ef0f8] /usr/bin/perl(Perl_call_sv+0x46c)[0x8078f0c] /usr/bin/perl[0x81005a0] /usr/bin/perl(Perl_sv_clear+0x41e)[0x8100b2e] /usr/bin/perl(Perl_sv_free2+0xe7)[0x8101177] /usr/bin/perl[0x81015d4] /usr/bin/perl[0x80f8164] /usr/bin/perl(Perl_sv_clean_objs+0x30)[0x8101860] /usr/bin/perl(perl_destruct+0x138)[0x807b598] /usr/bin/perl(main+0xfb)[0x805ed6b] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xf5)[0xb75d48c5] /usr/bin/perl[0x805ede1] ======= Memory map: ======== 08048000-081c1000 r-xp 00000000 fe:01 161742 /usr/bin/perl 081c1000-081c2000 r--p 00179000 fe:01 161742 /usr/bin/perl 081c2000-081c5000 rw-p 0017a000 fe:01 161742 /usr/bin/perl 09bbf000-0a049000 rw-p 00000000 00:00 0 [heap] b7319000-b7334000 r-xp 00000000 fe:01 464 /lib/i386-linux-gnu/libgcc_s.so.1 b7334000-b7335000 rw-p 0001a000 fe:01 464 /lib/i386-linux-gnu/libgcc_s.so.1 b7341000-b734c000 r-xp 00000000 fe:01 165131 /usr/lib/perl5/auto/Digest/SHA/SHA.so b734c000-b734d000 r--p 0000a000 fe:01 165131 /usr/lib/perl5/auto/Digest/SHA/SHA.so b734d000-b734e000 rw-p 0000b000 fe:01 165131 /usr/lib/perl5/auto/Digest/SHA/SHA.so b734e000-b735e000 r-xp 00000000 fe:01 262588 /usr/lib/perl/5.18.0/auto/POSIX/POSIX.so b735e000-b7360000 r--p 00010000 fe:01 262588 /usr/lib/perl/5.18.0/auto/POSIX/POSIX.so b7360000-b7361000 rw-p 00012000 fe:01 262588 /usr/lib/perl/5.18.0/auto/POSIX/POSIX.so b7361000-b7364000 r-xp 00000000 fe:01 262513 /usr/lib/perl/5.18.0/auto/Fcntl/Fcntl.so b7364000-b7365000 r--p 00002000 fe:01 262513 /usr/lib/perl/5.18.0/auto/Fcntl/Fcntl.so b7365000-b7366000 rw-p 00003000 fe:01 262513 /usr/lib/perl/5.18.0/auto/Fcntl/Fcntl.so b7366000-b736a000 r-xp 00000000 fe:01 262927 /usr/lib/perl/5.18.0/auto/IO/IO.so b736a000-b736b000 r--p 00003000 fe:01 262927 /usr/lib/perl/5.18.0/auto/IO/IO.so b736b000-b736c000 rw-p 00004000 fe:01 262927 /usr/lib/perl/5.18.0/auto/IO/IO.so b736c000-b7373000 r-xp 00000000 fe:01 957 /lib/i386-linux-gnu/i686/cmov/librt-2.17.so b7373000-b7374000 r--p 00006000 fe:01 957 /lib/i386-linux-gnu/i686/cmov/librt-2.17.so b7374000-b7375000 rw-p 00007000 fe:01 957 /lib/i386-linux-gnu/i686/cmov/librt-2.17.so b7377000-b737e000 r-xp 00000000 fe:01 262744 /usr/lib/perl/5.18.0/auto/Socket/Socket.so b737e000-b737f000 ---p 00007000 fe:01 262744 /usr/lib/perl/5.18.0/auto/Socket/Socket.so b737f000-b7380000 r--p 00007000 fe:01 262744 /usr/lib/perl/5.18.0/auto/Socket/Socket.so b7380000-b7381000 rw-p 00008000 fe:01 262744 /usr/lib/perl/5.18.0/auto/Socket/Socket.so b7381000-b7386000 r-xp 00000000 fe:01 263464 /usr/lib/perl/5.18.0/auto/Time/HiRes/HiRes.so b7386000-b7387000 r--p 00004000 fe:01 263464 /usr/lib/perl/5.18.0/auto/Time/HiRes/HiRes.so b7387000-b7388000 rw-p 00005000 fe:01 263464 /usr/lib/perl/5.18.0/auto/Time/HiRes/HiRes.so b7388000-b7588000 r--p 00000000 fe:01 133484 /usr/lib/locale/locale-archive b7588000-b7589000 rw-p 00000000 00:00 0 b7589000-b7592000 r-xp 00000000 fe:01 822 /lib/i386-linux-gnu/i686/cmov/libcrypt-2.17.so b7592000-b7593000 r--p 00008000 fe:01 822 /lib/i386-linux-gnu/i686/cmov/libcrypt-2.17.so b7593000-b7594000 rw-p 00009000 fe:01 822 /lib/i386-linux-gnu/i686/cmov/libcrypt-2.17.so b7594000-b75bb000 rw-p 00000000 00:00 0 b75bb000-b7764000 r-xp 00000000 fe:01 438 /lib/i386-linux-gnu/i686/cmov/libc-2.17.so b7764000-b7766000 r--p 001a9000 fe:01 438 /lib/i386-linux-gnu/i686/cmov/libc-2.17.so b7766000-b7767000 rw-p 001ab000 fe:01 438 /lib/i386-linux-gnu/i686/cmov/libc-2.17.so b7767000-b776a000 rw-p 00000000 00:00 0 b776a000-b7781000 r-xp 00000000 fe:01 754 /lib/i386-linux-gnu/i686/cmov/libpthread-2.17.so b7781000-b7782000 r--p 00016000 fe:01 754 /lib/i386-linux-gnu/i686/cmov/libpthread-2.17.so b7782000-b7783000 rw-p 00017000 fe:01 754 /lib/i386-linux-gnu/i686/cmov/libpthread-2.17.so b7783000-b7786000 rw-p 00000000 00:00 0 b7786000-b77c7000 r-xp 00000000 fe:01 169 /lib/i386-linux-gnu/i686/cmov/libm-2.17.so b77c7000-b77c8000 r--p 00040000 fe:01 169 /lib/i386-linux-gnu/i686/cmov/libm-2.17.so b77c8000-b77c9000 rw-p 00041000 fe:01 169 /lib/i386-linux-gnu/i686/cmov/libm-2.17.so b77c9000-b77cc000 r-xp 00000000 fe:01 386 /lib/i386-linux-gnu/i686/cmov/libdl-2.17.so b77cc000-b77cd000 r--p 00002000 fe:01 386 /lib/i386-linux-gnu/i686/cmov/libdl-2.17.so b77cd000-b77ce000 rw-p 00003000 fe:01 386 /lib/i386-linux-gnu/i686/cmov/libdl-2.17.so b77cf000-b77d0000 rw-p 00000000 00:00 0 b77d0000-b77d8000 r-xp 00000000 fe:01 264041 /usr/lib/perl/5.18.0/auto/Encode/Encode.so b77d8000-b77d9000 r--p 00007000 fe:01 264041 /usr/lib/perl/5.18.0/auto/Encode/Encode.so b77d9000-b77da000 rw-p 00008000 fe:01 264041 /usr/lib/perl/5.18.0/auto/Encode/Encode.so b77da000-b77dc000 rw-p 00000000 00:00 0 b77dc000-b77dd000 r-xp 00000000 00:00 0 [vdso] b77dd000-b77fc000 r-xp 00000000 fe:01 703 /lib/i386-linux-gnu/ld-2.17.so b77fc000-b77fd000 r--p 0001f000 fe:01 703 /lib/i386-linux-gnu/ld-2.17.so b77fd000-b77fe000 rw-p 00020000 fe:01 703 /lib/i386-linux-gnu/ld-2.17.so bfef4000-bff15000 rw-p 00000000 00:00 0 [stack] Search finished in 00:00, 0 matches found in 0% of a 32-bit space. The vanityhash program can be downloaded from http://www.finnie.org/software/vanityhash/vanityhash-1.1.tar.gz
  Thu Jun 20 18:17:36 2013 dom [...] cpan.org - Reference to ticket #82655 added
  Fri Jun 21 00:14:36 2013 mshelor [...] cpan.org - Taken
  Fri Jun 21 00:56:33 2013 mshelor [...] cpan.org - Correspondence added 30 min
The double-free appears to result from not resetting the internal SHA pointer to NULL once that structure is freed, having mistakenly assumed that this would be handled automatically by Perl's Safefree. I use the more cautious word 'appears' since the bug is difficult to reproduce reliably. By explicitly resetting the internal pointer to NULL in SHA.xs, I am unable to reproduce the double-free even sporadically. I can only surmise that Perl issues occasional redundant commands during object cleanup (triggered perhaps by asynchronous external events), leading to the observed behavior. Doing a hard reset of the pointer to the internal C structure in SHA.xs prevents Perl from attempting to free previously de-allocated structures. I'll release the fix in a few days pending further analysis and test. If you're interested, the trial patch is: diff -Naur 5.84/SHA.xs 5.85/SHA.xs --- 5.84/SHA.xs 2013-03-09 13:38:02.573017538 -0700 +++ 5.85/SHA.xs 2013-06-20 20:34:12.127040575 -0700 @@ -31,6 +31,9 @@ int shaclose(s) SHA * s +CODE: + RETVAL = shaclose(s); + sv_setiv(SvRV(ST(0)), 0); int shadump(file, s) Regards, Mark Shelor
  Fri Jun 21 00:56:33 2013 The RT System itself - Status changed from 'new' to 'open'
  Fri Jun 21 16:33:29 2013 dom [...] earth.li - Correspondence added
Subject: Re: [rt.cpan.org #86295] Another double free crash
Date: Fri, 21 Jun 2013 21:33:15 +0100
To: Mark Shelor via RT <bug-Digest-SHA [...] rt.cpan.org>
From: Dominic Hargreaves <dom [...] earth.li>
On Fri, Jun 21, 2013 at 12:56:33AM -0400, Mark Shelor via RT wrote: Show quoted text
> The double-free appears to result from not resetting the internal SHA pointer to NULL once that structure is freed, having mistakenly assumed that this would be handled automatically by Perl's Safefree. I use the more cautious word 'appears' since the bug is difficult to reproduce reliably. > > By explicitly resetting the internal pointer to NULL in SHA.xs, I am unable to reproduce the double-free even sporadically. I can only surmise that Perl issues occasional redundant commands during object cleanup (triggered perhaps by asynchronous external events), leading to the observed behavior. Doing a hard reset of the pointer to the internal C structure in SHA.xs prevents Perl from attempting to free previously de-allocated structures. > > I'll release the fix in a few days pending further analysis and test. If you're interested, the trial patch is: > > diff -Naur 5.84/SHA.xs 5.85/SHA.xs > --- 5.84/SHA.xs 2013-03-09 13:38:02.573017538 -0700 > +++ 5.85/SHA.xs 2013-06-20 20:34:12.127040575 -0700 > @@ -31,6 +31,9 @@ > int > shaclose(s) > SHA * s > +CODE: > + RETVAL = shaclose(s); > + sv_setiv(SvRV(ST(0)), 0); > > int > shadump(file, s) > > > Regards, Mark Shelor
Thanks for the prompt response! This appears, as far as I can tell from a few dozen runs, to have fixed this for me. Look forward to the new release, at which point I'll think about pushing this fix into Debian. Cheers, Dominic.
  Wed Jun 26 10:30:06 2013 mshelor [...] cpan.org - Correspondence added 120 min
Version 5.85 provides a workaround to the double-free problem by setting Perl's internal SHA pointer to NULL once that structure has been deallocated. Such a solution is highly irregular, but appears to be effective. Long-duration stress tests of the workaround demonstrate robust behavior when generating asynchronous interrupts at both systematically-varying and randomly-varying time intervals. This is being called a workaround rather than a fix because there's nothing in the Digest::SHA code or vanityhash code which should cause the deallocation routine (shaclose) to be reentered once the object's SHA structure has been freed. I suspect, but cannot yet prove, that the reentry results from the object's destructor (DESTROY) being invoked multiple times by Perl, which of course should not happen. As Larry says: "Explicitly calling DESTROY is possible but seldom needed. It might even be harmful since running the destructor more than once on the same object could prove unpleasant." It's possible that some sort of duplication and resulting race condition could occur when large numbers of SHA objects are being DESTROYed as a result of asynchronous interrupts. This might even suggest why double-free problems appear to occur so frequently in Perl. It appears drastic to directly manipulate the value of internal pointers set up by Perl, but such a workaround is practically effective since it enables a reasonably tight guard to be set up against repeated deallocations. Even though the stress tests can't trigger double-frees deterministically, they are effective in statistically generating them several times a minute when running earlier versions of Digest::SHA. With the workaround in 5.85, not a single double-free was witnessed during hours of sustained testing.
  Wed Jun 26 10:30:07 2013 mshelor [...] cpan.org - Status changed from 'open' to 'resolved'
  Wed Jun 26 10:30:07 2013 mshelor [...] cpan.org - Fixed in 5.85 added