Bug #3227 for PAR: Errors while deleting temp directory

Tue Aug 12 23:48:34 2003 astewart1 [...] cox.net - Ticket created

From:	"Alan Stewart" <astewart1 [...] cox.net>
To:	bug-par [...] rt.cpan.org
Date:	Tue, 12 Aug 2003 20:48:24 -0700
Subject:	Errors while deleting temp directory

Using PAR-0.73 on NT 4.0/sp6. I get occasional "unhandled exception" crashes when launching the compiled a.exe by different means. The VC debugger indicates the crash occurs in par_rmtmpdir in par.exe. It appears that both a.exe (static.c) and par.exe (main.c) invoke this subroutine near their end. I don't know why that would cause a crash, and only when invoked from some places, but it's redundant. When I eliminated the call in main.c, leaving the one in static.c, everything works fine. The call in main.c can't delete everything anyway, since some of the files are DLLs that are still in use. This happens when compiling with "pp -d" where the only call to par_rmtmpdir would be in main.c. Why not use the same process for "-d" of having a.exe extract par.exe but not extract perlxx.dll, and clean up after par.exe at the end of a.exe? main.c has a #ifndef PAR_MKTMPDIR but I don't see a define elsewhere. Was there supposed to be a define in the makefile? Alan Stewart

Wed Aug 13 00:03:41 2003 autrijus [...] autrijus.org - Correspondence added

Date:	Wed, 13 Aug 2003 12:03:34 +0800
From:	Autrijus Tang <autrijus [...] autrijus.org>
To:	Alan Stewart via RT <bug-PAR [...] rt.cpan.org>
CC:	"AdminCc of cpan Ticket #3227": ;
Subject:	Re: [cpan #3227] Errors while deleting temp directory
RT-Send-Cc:

On Tue, Aug 12, 2003 at 11:48:35PM -0400, Alan Stewart via RT wrote: Show quoted text

> The call in main.c can't delete everything anyway, since some of the files are > DLLs that are still in use. This happens when compiling with "pp -d" where the > only call to par_rmtmpdir would be in main.c. Why not use the same process for > "-d" of having a.exe extract par.exe but not extract perlxx.dll, and clean up > after par.exe at the end of a.exe?

Can you explain a bit better why this would be a gain over the current way in "pp -d"? I agree that main.c should not attempt to rmdir when it's spawned by static.c. Can you help adding a test to !getenv("PAR_ARGC") before par_rmtmpdir(), and see if it suffices? Show quoted text

> main.c has a > #ifndef PAR_MKTMPDIR > but I don't see a define elsewhere. Was there supposed to be a define in the > makefile?

It's there in main.c line 7. Thanks, /Autrijus/

Download (untitled)
application/pgp-signature 187b

Message body not shown because it is not plain text.

Wed Aug 13 12:48:43 2003 Guest - Correspondence added

[autrijus@autrijus.org - Wed Aug 13 00:03:41 2003]: Show quoted text

> Can you explain a bit better why this would be a gain over the current > way in "pp -d"? >

When main.c isn't spawned from static.c for "-d", it can't delete the DLLs it uses, so the directory is also not deleted. This is the same problem with earlier versions. I thought that was why you created the static.c approach, so main.c can end and then the DLLs can be cleaned up. Can't the same thing be done whether you are extracting perlxx.dll or using the installed perl one? Show quoted text

>

> > main.c has a > > #ifndef PAR_MKTMPDIR > > but I don't see a define elsewhere. Was there supposed to be a

> define in the

> > makefile?

> > It's there in main.c line 7. >

True, but that's right there after the #ifndef at line 6 and nowhere else that I see, so it's really unconditional. I thought it might have been intended as a conditional at compile time for main.c rather than an %ENV check at run time as you suggest above.

Wed Aug 13 13:19:57 2003 autrijus [...] autrijus.org - Correspondence added

Date:	Thu, 14 Aug 2003 01:19:47 +0800
From:	Autrijus Tang <autrijus [...] autrijus.org>
To:	Guest via RT <bug-PAR [...] rt.cpan.org>
CC:	"AdminCc of cpan Ticket #3227": ;
Subject:	Re: [cpan #3227] Errors while deleting temp directory
RT-Send-Cc:

On Wed, Aug 13, 2003 at 12:48:44PM -0400, Guest via RT wrote: Show quoted text

> > Can you explain a bit better why this would be a gain over the current > > way in "pp -d"?

> When main.c isn't spawned from static.c for "-d", it can't delete the > DLLs it uses, so the directory is also not deleted. This is the same > problem with earlier versions. I thought that was why you created the > static.c approach, so main.c can end and then the DLLs can be cleaned > up. Can't the same thing be done whether you are extracting perlxx.dll > or using the installed perl one?

Ahh, I see. You are not talking about perl5x.dll, but other XS files. I really wonder if there's a way to not add a process spawn overhead and be able to delete the DLL files. Is there ways to explicitly close all opened DLL files on Win32? Or PERL_DESTRUCT already does that? Show quoted text

> > It's there in main.c line 7.

> True, but that's right there after the #ifndef at line 6 and nowhere > else that I see, so it's really unconditional. I thought it might have > been intended as a conditional at compile time for main.c rather than > an %ENV check at run time as you suggest above.

Right. I should remove the unconditional flag right away. Thanks, /Autirjus/

Download (untitled)
application/pgp-signature 187b

Message body not shown because it is not plain text.

Thu Aug 14 22:15:04 2003 Guest - Correspondence added

From:

Alan Stewart

[autrijus@autrijus.org - Wed Aug 13 13:19:57 2003]: Show quoted text

> > I really wonder if there's a way to not add a process spawn overhead > and be able to delete the DLL files. Is there ways to explicitly

close Show quoted text

> all opened DLL files on Win32? Or PERL_DESTRUCT already does that? >

I see two API functions that might be useful: GetModuleHandleEx gets the handle to a DLL given the file name. FreeLibrary decrements the reference count to the DLL. Using these on all the DLL files you know have been extracted might free the process so par_rmtmpdir can work. I have no idea if the Perl core tries to do anything like this already. I'll do some searching thru the source for these APIs. But, I am not an C/Win32 API programmer. . .

Sat Aug 16 00:05:58 2003 Guest - Correspondence added

From:

Alan Stewart

[guest - Thu Aug 14 22:15:04 2003]: Show quoted text

> [autrijus@autrijus.org - Wed Aug 13 13:19:57 2003]: >

> > > > I really wonder if there's a way to not add a process spawn overhead > > and be able to delete the DLL files. Is there ways to explicitly

> close

> > all opened DLL files on Win32? Or PERL_DESTRUCT already does that? > >

>

I have a working mktmpdir.c that uses GetModuleHandle and FreeLibrary to decrement the references from the loaded DLLs to the extracted files (see attached). All files now unlink with "pp -d", but I have not tried to eliminate the use of spawning with plain "pp". Perl core code does a LoadLibrary, but not a FreeLibrary anywhere. Each DLL is probably only loaded once, but the MSDN says multiple loads are possible, so I decrement until none. MSDN also says to prefer GetModuleHandleEx, but that is not supported in all Windows versions. I tested with the latest snap. BTW, the snap makefile.PL requires ScanDeps version 0.26, but CPAN only has 0.25 ! Alan Stewart

Download mktmpdir.zip
application/x-zip-compressed 1.8k

Message body not shown because it is not plain text.

Sat Aug 16 03:02:31 2003 autrijus [...] autrijus.org - Correspondence added

Date:	Sat, 16 Aug 2003 15:01:54 +0800
From:	Autrijus Tang <autrijus [...] autrijus.org>
To:	Guest via RT <bug-PAR [...] rt.cpan.org>
CC:	"AdminCc of cpan Ticket #3227": ;
Subject:	Re: [cpan #3227] Errors while deleting temp directory
RT-Send-Cc:

On Sat, Aug 16, 2003 at 12:05:59AM -0400, Guest via RT wrote: Show quoted text

> I have a working mktmpdir.c that uses GetModuleHandle and FreeLibrary > to decrement the references from the loaded DLLs to the extracted files > (see attached). All files now unlink with "pp -d",

Yay! Your patch is applied; I'll roll a new release as soon as I get around to test it a bit on Win32. Show quoted text

> but I have not tried to eliminate the use of spawning with plain "pp".

That will be much appreciated. I wonder how would it work, though -- is it as simple as a LoadLibrary() call to the extracted perl5x.dll? Show quoted text

> I tested with the latest snap. BTW, the snap makefile.PL requires > ScanDeps version 0.26, but CPAN only has 0.25 !

0.27 is now uploaded. :-) Thanks, /Autrijus/

Download (untitled)
application/pgp-signature 187b

Message body not shown because it is not plain text.

Sat Aug 16 12:49:33 2003 Guest - Correspondence added

[autrijus@autrijus.org - Sat Aug 16 03:02:31 2003]: Show quoted text

> That will be much appreciated. I wonder how would it work, though -- > is it as simple as a LoadLibrary() call to the extracted perl5x.dll? >

Looks like static.c should become just a copy of main.c, with the addition of the WRITE_load_me_0 code and a LoadLibrary(my_file). And whatever needs to change in the Makefile and pp. Alan Stewart

Wed Aug 20 21:15:06 2003 Guest - Correspondence added

From:

Alan Stewart

[guest - Sat Aug 16 12:49:33 2003]: Show quoted text

> [autrijus@autrijus.org - Sat Aug 16 03:02:31 2003]: >

> > That will be much appreciated. I wonder how would it work, though

> --

> > is it as simple as a LoadLibrary() call to the extracted perl5x.dll? > >

> > Looks like static.c should become just a copy of main.c, with the > addition of the WRITE_load_me_0 code and a LoadLibrary(my_file). > > And whatever needs to change in the Makefile and pp. > > Alan Stewart

A little studying and it looks a little less simple, but not bad. I think it needs to do: 1. Not link static.c with perl5x.lib (Makefile.PL hack?) so the .exe doesn't try to load perl5x.dll on startup, before we have a chance to extract it. 2. WRITE_loadme_0 and LoadLibrary('/path/perl5x.dll'). 3. For each perl5x function needed, GetProcAddress('Perl_func_name') to get the function address into a variable with the same name as the Perl API function. 4. Let the compiler notice the difference between a real function call and an indirect call with the same syntax for the remainder of the code copied from main.c. Looks like there are only 36 Perl API functions useed directly or indirectly from main.c out of the 1100+ possible ones in perl5x.dll. VC++ 6.0 and later has a linker option /DELAYLOAD:perl56.dll that would simplify this greatly by modding the .exe link table and adding a helper function to do essentially what I said above, but that would restrict the compiler anyone could use. I'll give this a shot this weekend.... On the Unix/Linux side of things, does the shared library get linked at load time or is there an on-demand option that gives you a chance to do the extraction before the first function call? Alan Stewart

Fri Apr 30 12:53:34 2004 autrijus [...] cpan.org - Status changed from 'new' to 'resolved'