Skip Menu |

This queue is for tickets about the Data-Clone CPAN distribution.

Report information
The Basics
Id: 92570
Status: open
Priority: 0/
Queue: Data-Clone

People
Owner: Nobody in particular
Requestors: 'spro^^*%*^6ut# [...] &$%*c
KENTNL [...] cpan.org
Cc: BULKDD [...] cpan.org
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: 0.004



Subject: [PATCH] Handle stack reallocation
See the attached patch. You should be able to feed it straight to ‘git am’.
Subject: patch.text
From: Father Chrysostomos <sprout@cpan.org> Handle stack reallocation in callbacks ST expands to something referencing PL_stack_base. Since the order of evalution of a=b is undefined in C, the value of PL_stack_base may be read before sv_clone is called. sv_clone may reallocate the stack, so the value gets written to freed memory and clone() returns its argu- ment instead of the clone. diff -Nurp Data-Clone-0.003-zBXsq6-orig/Data-Clone.xs Data-Clone-0.003-zBXsq6/Data-Clone.xs --- Data-Clone-0.003-zBXsq6-orig/Data-Clone.xs 2011-01-15 05:58:49.000000000 -0800 +++ Data-Clone-0.003-zBXsq6/Data-Clone.xs 2014-01-29 21:26:23.000000000 -0800 @@ -392,7 +392,8 @@ void clone(SV* sv) CODE: { - ST(0) = sv_clone(sv); + sv = sv_clone(sv); + ST(0) = sv; XSRETURN(1); } diff -Nurp Data-Clone-0.003-zBXsq6-orig/MANIFEST Data-Clone-0.003-zBXsq6/MANIFEST --- Data-Clone-0.003-zBXsq6-orig/MANIFEST 2011-01-15 05:53:37.000000000 -0800 +++ Data-Clone-0.003-zBXsq6/MANIFEST 2014-01-29 21:25:25.000000000 -0800 @@ -28,6 +28,7 @@ t/03_scalar_ref.t t/04_tree.t t/05_super.t t/06_tie.t +t/07_stack.t t/10_threads.t t/11_leaktrace.t xshelper.h diff -Nurp Data-Clone-0.003-zBXsq6-orig/t/07_stack.t Data-Clone-0.003-zBXsq6/t/07_stack.t --- Data-Clone-0.003-zBXsq6-orig/t/07_stack.t 1969-12-31 16:00:00.000000000 -0800 +++ Data-Clone-0.003-zBXsq6/t/07_stack.t 2014-01-29 21:25:09.000000000 -0800 @@ -0,0 +1,22 @@ +#!perl -w + +use strict; +use warnings FATAL => 'all'; + +use Test::More; + +use Data::Clone; + +{ + package Bar; + sub clone { + () = (1)x100000; # extend the stack + return [] + } +} + +my $before = bless [], Bar::; +my $after = clone($before); +isn't $after, $before, 'stack reallocation during callback'; + +done_testing;
On Thu Jan 30 00:38:32 2014, SPROUT wrote: Show quoted text
> See the attached patch. You should be able to feed it straight to ‘git am’.
-------------------------------------------------------------- ST expands to something referencing PL_stack_base. Since the order of evalution of a=b is undefined in C, the value of PL_stack_base may be read before sv_clone is called. sv_clone may reallocate the stack, so the value gets written to freed memory and clone() returns its argu- ment instead of the clone. -------------------------------------------------------------- On all compilers or just GCC? Below is from XS_Data__Clone_clone compile with Visual C 2003 in -O1. esi is my_perl, edi is ax. -------------------------------------------------------------- mov edi, eax shl edi, 2 push dword ptr [ecx+edi] ; sv push esi ; my_perl call _Data_Clone_sv_clone pop ecx pop ecx mov ecx, [esi+interpreter.Istack_base] mov [edi+ecx], eax mov eax, [esi+interpreter.Istack_base] add eax, edi pop edi mov [esi], eax pop esi retn ---------------------------------------------------------------
On Thu Jan 30 02:19:59 2014, BULKDD wrote: Show quoted text
> On Thu Jan 30 00:38:32 2014, SPROUT wrote:
> > See the attached patch. You should be able to feed it straight to > > ‘git am’.
> > > -------------------------------------------------------------- > ST expands to something referencing PL_stack_base. Since the order of > evalution of a=b is undefined in C, the value of PL_stack_base may be > read before sv_clone is called. sv_clone may reallocate the stack, so > the value gets written to freed memory and clone() returns its argu- > ment instead of the clone. > -------------------------------------------------------------- > On all compilers or just GCC?
GCC and clang. Show quoted text
> Below is from XS_Data__Clone_clone > compile with Visual C 2003 in -O1. esi is my_perl, edi is ax. > -------------------------------------------------------------- > mov edi, eax > shl edi, 2 > push dword ptr [ecx+edi] ; sv > push esi ; my_perl > call _Data_Clone_sv_clone > pop ecx > pop ecx > mov ecx, [esi+interpreter.Istack_base] > mov [edi+ecx], eax > mov eax, [esi+interpreter.Istack_base] > add eax, edi > pop edi > mov [esi], eax > pop esi > retn > ---------------------------------------------------------------
I am not familiar with assembler. Even if that shows that Data::Clone works under VC, my patch is still valid, is it not?
On Fri Jan 31 01:38:16 2014, SPROUT wrote: Show quoted text
> On Thu Jan 30 02:19:59 2014, BULKDD wrote:
> > On Thu Jan 30 00:38:32 2014, SPROUT wrote:
> > > See the attached patch. You should be able to feed it straight to > > > ‘git am’.
> > > > > > -------------------------------------------------------------- > > ST expands to something referencing PL_stack_base. Since the order of > > evalution of a=b is undefined in C, the value of PL_stack_base may be > > read before sv_clone is called. sv_clone may reallocate the stack, so > > the value gets written to freed memory and clone() returns its argu- > > ment instead of the clone. > > -------------------------------------------------------------- > > On all compilers or just GCC?
> > GCC and clang. >
> > Below is from XS_Data__Clone_clone > > compile with Visual C 2003 in -O1. esi is my_perl, edi is ax. > > -------------------------------------------------------------- > > mov edi, eax > > shl edi, 2 > > push dword ptr [ecx+edi] ; sv > > push esi ; my_perl > > call _Data_Clone_sv_clone > > pop ecx > > pop ecx > > mov ecx, [esi+interpreter.Istack_base] > > mov [edi+ecx], eax > > mov eax, [esi+interpreter.Istack_base] > > add eax, edi > > pop edi > > mov [esi], eax > > pop esi > > retn > > ---------------------------------------------------------------
> > I am not familiar with assembler. Even if that shows that Data::Clone > works under VC, my patch is still valid, is it not?
The bigger problem is, you are patching a very common xsubpp idiom. You need to fix GCC, not work around a code gen bug that is everywhere. Here is my example. ---------------------------------- //user supplied code ST(0) = sv_newmortal(); sv_setiv(ST(0), RETVAL); XSRETURN(1); ---------------------------------- here is a real example, notice the part that is user supplied based on cpp tokens ----------------------------------- XS_EUPXS(XS_XS__APItest_peep_record) { dVAR; dXSARGS; if (items != 0) croak_xs_usage(cv, ""); { #line 3144 "APItest.xs" dMY_CXT; #line 5122 "APItest.c" SV * RETVAL; #line 3146 "APItest.xs" /* What if newRV_inc (which can be any C func the user put in CODE:) realloced the stack? */ RETVAL = newRV_inc((SV *)MY_CXT.peep_recorder); #line 5126 "APItest.c" ST(0) = RETVAL; sv_2mortal(ST(0)); } XSRETURN(1); } --------------------------------------- Using A Win32 W64 GCC 4.6.3 from some strawberry perl. --------------------------------------- gcc -c "-I." -s -O2 -DWIN32 -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -fno-strict-aliasing -mms-bitfields -Wall -W -Wno-comment -s -O2 -DVERSION=\"0.003\" -DXS_VERSION=\"0.003\" -o Data-Clone.o "-IC:\sp 3216\perl\lib\CORE" -DUSE_PPPORT Data-Clone.c Data-Clone.xs: In function 'my_dopoptosub_at': Data-Clone.xs:109:18: warning: unused parameter 'my_perl' [-Wunused-parameter] Data-Clone.xs: In function 'dc_need_to_call': Data-Clone.xs:191:17: warning: unused parameter 'my_perl' [-Wunused-parameter] Running Mkbootstrap for Data::Clone () --------------------------------------- lea esi, ds:4[ecx*4] ; ax* 4 add edi, esi ; add scaled ax to PL_stack_base mov eax, [edi] ; deref SV ** mov [esp+1Ch+var_1C], ebx ; copy my_perl to cstack mov [esp+1Ch+var_18], eax ; copy sv to c stack call Data_Clone_sv_clone ; call function mov [edi], eax ; assign return value of Data_Clone_sv_clone to deref of perl stack, this is the SEGV if stack was realloced, since PL_stack_base was NOT reread, this is "ST(0) = retsv" basically ;now starts XS_RETURN(1) add esi, [ebx+0Ch] ; ebx is my_perl, 0x0C is PL_stack_base, so add PL_stack_base to scaled ax mov [ebx], esi ; assign new SV ** to PL_stack_sp, since stack_sp is 1st member of interp stuct, no offset is specificed mov ebx, [esp+1Ch+var_C]; restore non-vol reg mov esi, [esp+1Ch+var_8]; restore non-vol reg mov edi, [esp+1Ch+var_4]; restore non-vol reg add esp, 1Ch ; wipe out C stack space retn ; return to caller function [whose address is on c stack at register esp] ---------------------------------------
On Fri Jan 31 15:04:26 2014, BULKDD wrote: Show quoted text
> The bigger problem is, you are patching a very common xsubpp idiom. > You need to fix GCC, not work around a code gen bug
It’s not a bug, because the C standard guarantees no particular order in which the operands of an assignment are executed. Show quoted text
> that is > everywhere. Here is my example. > ---------------------------------- > //user supplied code > ST(0) = sv_newmortal(); > sv_setiv(ST(0), RETVAL); > XSRETURN(1); > ---------------------------------- > here is a real example, notice the part that is user supplied based on > cpp tokens > ----------------------------------- > XS_EUPXS(XS_XS__APItest_peep_record) > { > dVAR; dXSARGS; > if (items != 0) > croak_xs_usage(cv, ""); > { > #line 3144 "APItest.xs" > dMY_CXT; > #line 5122 "APItest.c" > SV * RETVAL; > #line 3146 "APItest.xs" > /* What if newRV_inc (which can be any C func the user put in CODE:) > realloced the stack? */
That will not cause any problem, because the ST(0) is in a separate statement. Show quoted text
> RETVAL = newRV_inc((SV *)MY_CXT.peep_recorder); > #line 5126 "APItest.c" > ST(0) = RETVAL; > sv_2mortal(ST(0)); > } > XSRETURN(1); > } > --------------------------------------- > Using A Win32 W64 GCC 4.6.3 from some strawberry perl. > --------------------------------------- > gcc -c "-I." -s -O2 -DWIN32 -DPERL_TEXTMODE_SCRIPTS > -DPERL_IMPLICIT_CONTEXT > -DPERL_IMPLICIT_SYS -fno-strict-aliasing -mms-bitfields -Wall -W -Wno- > comment -s > -O2 -DVERSION=\"0.003\" -DXS_VERSION=\"0.003\" -o Data-Clone.o > "-IC:\sp > 3216\perl\lib\CORE" -DUSE_PPPORT Data-Clone.c > Data-Clone.xs: In function 'my_dopoptosub_at': > Data-Clone.xs:109:18: warning: unused parameter 'my_perl' [-Wunused- > parameter] > Data-Clone.xs: In function 'dc_need_to_call': > Data-Clone.xs:191:17: warning: unused parameter 'my_perl' [-Wunused- > parameter] > Running Mkbootstrap for Data::Clone () > --------------------------------------- > lea esi, ds:4[ecx*4] ; ax* 4 > add edi, esi ; add scaled ax to PL_stack_base > mov eax, [edi] ; deref SV ** > mov [esp+1Ch+var_1C], ebx ; copy my_perl to cstack > mov [esp+1Ch+var_18], eax ; copy sv to c stack > call Data_Clone_sv_clone ; call function > mov [edi], eax ; assign return value of Data_Clone_sv_clone to > deref of perl stack, this is the SEGV if stack was realloced, since > PL_stack_base was NOT reread, this is "ST(0) = retsv" basically > ;now starts XS_RETURN(1) > add esi, [ebx+0Ch] ; ebx is my_perl, 0x0C is PL_stack_base, so add > PL_stack_base to scaled ax > mov [ebx], esi ; assign new SV ** to PL_stack_sp, since stack_sp > is 1st member of interp stuct, no offset is specificed > mov ebx, [esp+1Ch+var_C]; restore non-vol reg > mov esi, [esp+1Ch+var_8]; restore non-vol reg > mov edi, [esp+1Ch+var_4]; restore non-vol reg > add esp, 1Ch ; wipe out C stack space > retn ; return to caller function [whose address is on c stack at > register esp] > ---------------------------------------
Thank you for the explanation.

Seems the original code in question has a memory write issue. ( see attached valgrind segment )

And it seems that is is causing crashes for HTML::FormHandler on >5.19.5   ( See above linked )

Applying FatherC's patch resolves the memory problem.

Subject: clone.txt
==907262== Invalid write of size 8 ==907262== at 0x108F3168: XS_Data__Clone_clone (Data-Clone.xs:395) ==907262== by 0x4F5784: Perl_pp_entersub (pp_hot.c:2767) ==907262== by 0x4C2DC9: Perl_runops_debug (dump.c:2420) ==907262== by 0x44BBC4: perl_run (perl.c:2449) ==907262== by 0x4229E4: main (perlmain.c:112) ==907262== Address 0x5d37d20 is 16 bytes inside a block of size 1,024 free'd ==907262== at 0x4C2C01E: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==907262== by 0x4C564D: Perl_safesysrealloc (util.c:244) ==907262== by 0x4E6E9C: Perl_av_extend_guts (av.c:154) ==907262== by 0x54322E: Perl_stack_grow (scope.c:38) ==907262== by 0x581599: Perl_do_kv (in /home/kent/perl5/perlbrew/perls/perl-5.19.9/bin/perl5.19.9) ==907262== by 0x4ED366: Perl_pp_rv2av (pp_hot.c:936) ==907262== by 0x4C2DC9: Perl_runops_debug (dump.c:2420) ==907262== by 0x4428E9: Perl_call_sv (perl.c:2749) ==907262== by 0x108F1E8A: dc_call_sv1 (Data-Clone.xs:176) ==907262== by 0x108F214B: dc_clone_object (Data-Clone.xs:227) ==907262== by 0x108F2B3D: clone_sv (Data-Clone.xs:266) ==907262== by 0x108F2FF1: Data_Clone_sv_clone (Data-Clone.xs:341)
Father C's patch has been applied to Data::Clone 0.004 to resolve this ticket