Bug #11139 for CPANPLUS: Calls to tar/gzip lock up on Windows

Thu Jan 20 18:26:44 2005 Guest - Ticket created

Subject:

Calls to tar/gzip lock up on Windows

Testing Locale-Maketext-TieHash-nbsp-0.02 on Windows causes CPANPLUS to lock up, waiting for results from tar or gzip: CPAN Terminal> t Locale-Maketext-TieHash-nbsp-0.02 [MSG] Checking if source files are up to date [MSG] Retrieving C:\Documents and Settings\Rob.MAGNETO-XP\Application Data\.cpan plus\sourcefiles.2.13.stored Testing Locale::Maketext::TieHash::nbsp [MSG] Trying to get 'ftp://cpan.etla.org/pub/CPAN/authors/id/S/ST/STEFFENW/CHECK SUMS' [MSG] Checksum matches for 'Locale-Maketext-TieHash-nbsp-0.02.tar.gz' Could not open file 'C:\Documents and Settings\Rob.MAGNETO-XP\Application Data\. cpanplus\5.8.6\build\Locale-Maketext-TieHash-nbsp-0.02\manifest': Permission den ied at C:/Perl/site/lib/CPANPLUS/Internals/Extract.pm line 158 Could not extract 'Locale-Maketext-TieHash-nbsp-0.02/manifest' at C:/Perl/site/l ib/CPANPLUS/Internals/Extract.pm line 158 Unable to extract 'C:\Documents and Settings\Rob.MAGNETO-XP\Application Data\.cp anplus\authors\id\S\ST\STEFFENW\Locale-Maketext-TieHash-nbsp-0.02.tar.gz': Could not extract 'Locale-Maketext-TieHash-nbsp-0.02/manifest' at C:/Perl/site/lib/CP ANPLUS/Module.pm line 469 C:\Perl\bin\tar.EXE: Cannot open -: No such file or directory C:\Perl\bin\tar.EXE: Error is not recoverable: exiting now gzip.EXE: stdout: Invalid argument Ctrl-Break is the only way to end it. Manually, Makefile.PL and nmake test work fine for this module. So what is CPANPLUS doing differently?

Sat Jan 22 06:28:30 2005 kane [...] cpan.org - Correspondence added

[guest - Thu Jan 20 18:26:44 2005]: Show quoted text

> Testing Locale-Maketext-TieHash-nbsp-0.02 on Windows causes CPANPLUS > to lock up, waiting for results from tar or gzip:

If that's correct, all tests you do should lock up waiting for results from tar or gzip... Show quoted text

> CPAN Terminal> t Locale-Maketext-TieHash-nbsp-0.02 > > [MSG] Checking if source files are up to date > [MSG] Retrieving C:\Documents and Settings\Rob.MAGNETO-XP\Application > Data\.cpan > plus\sourcefiles.2.13.stored > Testing Locale::Maketext::TieHash::nbsp > [MSG] Trying to get > 'ftp://cpan.etla.org/pub/CPAN/authors/id/S/ST/STEFFENW/CHECK > SUMS' > [MSG] Checksum matches for 'Locale-Maketext-TieHash-nbsp-0.02.tar.gz' > Could not open file 'C:\Documents and Settings\Rob.MAGNETO- > XP\Application Data\. > cpanplus\5.8.6\build\Locale-Maketext-TieHash-nbsp-0.02\manifest': > Permission den > ied at C:/Perl/site/lib/CPANPLUS/Internals/Extract.pm line 158 > Could not extract 'Locale-Maketext-TieHash-nbsp-0.02/manifest' at > C:/Perl/site/l > ib/CPANPLUS/Internals/Extract.pm line 158 > Unable to extract 'C:\Documents and Settings\Rob.MAGNETO- > XP\Application Data\.cp > anplus\authors\id\S\ST\STEFFENW\Locale-Maketext-TieHash-nbsp- > 0.02.tar.gz': Could > not extract 'Locale-Maketext-TieHash-nbsp-0.02/manifest' at > C:/Perl/site/lib/CP > ANPLUS/Module.pm line 469

ehm, permission denied? are you sure your permissions (or the ones in the archive you are trying to extract) are set correctly? Show quoted text

> C:\Perl\bin\tar.EXE: Cannot open -: No such file or directory > C:\Perl\bin\tar.EXE: Error is not recoverable: exiting now > > gzip.EXE: stdout: Invalid argument

apparently windows can not deal with piped commands, in this case: gzip -d .... | tar -xvf - can you verify that this is indeed the case? Show quoted text

> Manually, Makefile.PL and nmake test work fine for this module. So > what is CPANPLUS doing differently?

This has nothing to do with makefile.pl/nmake, but iwth extracting the modules... using Archive::Tar is your safest bet here, and of course making sure the permissions on your extract dir + archives are set properly

Sat Jan 22 09:19:21 2005 Guest - Correspondence added

Subject:	Re: Calls to tar/gzip lock up on Windows
From:	rrwo [...] cpan.org

I seem to be having the same problem with the following: Math-SymbolicX-NoSimplification-0.01 But most other modules work fine. On 22/01/2005 11:29 via RT wrote: Show quoted text

>> Data\.cp

anplus\authors\id\S\ST\STEFFENW\Locale-Maketext-TieHash-nbsp- 0.02.tar.gz': Could not extract Show quoted text

>> 'Locale-Maketext-TieHash-nbsp-0.02/manifest' at C:/Perl/site/lib/CP >> ANPLUS/Module.pm line 469

> > > > ehm, permission denied? are you sure your permissions (or the ones in > the archive you are trying to extract) are set correctly?

There is no permissions problem in the .cpanplus directories. I can manually go there and extract archives, run make etc. with no problems. I can also go to the build directory, where the files appear to be properly extracted, run the tests, etc. Also no problems. And I've configured CPANPLUS to prefer Perl modules over binaries, so I don't know why it's calling gzip and tar. Show quoted text

> apparently windows can not deal with piped commands, in this case:

gzip -d .... | tar -xvf - Show quoted text

> > can you verify that this is indeed the case?

No, it doesn't work because that's the wrong set of options for gzip. This works, however: gzip -dc Math-SymbolicX-NoSimplification-0.01.tar.gz |tar -xvf - Without the 'c' option, it operates directly on file and sends nothing to tar. Show quoted text

> This has nothing to do with makefile.pl/nmake, but iwth extracting > the modules... > > using Archive::Tar is your safest bet here, and of course making sure > the permissions on your extract dir + archives are set properly

I have Archive::Tar installed (I've even just upgraded to the latest version, but it does not help). And the permissions are fine.

Sat Jan 22 09:23:37 2005 kane [...] dwim.org - Correspondence added

From:	"Jos I. Boumans" <kane [...] dwim.org>
Subject:	Re: [cpan #11139] Re: Calls to tar/gzip lock up on Windows
Date:	Sat, 22 Jan 2005 15:19:25 +0100
To:	bug-CPANPLUS [...] rt.cpan.org
RT-Send-Cc:

On Jan 22, 2005, at 3:19 PM, Guest via RT wrote: Show quoted text

> > This message about CPANPLUS was sent to you by guest <> via rt.cpan.org > > Full context and any attached attachments can be found at: > <URL: https://rt.cpan.org/Ticket/Display.html?id=11139 > > > > I seem to be having the same problem with the following: > > Math-SymbolicX-NoSimplification-0.01 > > But most other modules work fine. > > On 22/01/2005 11:29 via RT wrote: >

>>> Data\.cp

> anplus\authors\id\S\ST\STEFFENW\Locale-Maketext-TieHash-nbsp- > 0.02.tar.gz': Could not extract

>>> 'Locale-Maketext-TieHash-nbsp-0.02/manifest' at C:/Perl/site/lib/CP >>> ANPLUS/Module.pm line 469

>> >> >> >> ehm, permission denied? are you sure your permissions (or the ones in >> the archive you are trying to extract) are set correctly?

> > > There is no permissions problem in the .cpanplus directories. I can > manually go there and extract archives, run make etc. with no problems.

ok, then why is there a permission denied coming from the extraction process? there is /something/ lacking permissions /somehwere/ Show quoted text

> I can also go to the build directory, where the files appear to be > properly extracted, run the tests, etc. Also no problems. > > And I've configured CPANPLUS to prefer Perl modules over binaries, so I > don't know why it's calling gzip and tar.

If the perl module fails it will attempt the alternative.. in this case tar + gzip. Show quoted text

>> apparently windows can not deal with piped commands, in this case:

> gzip -d .... | tar -xvf -

>> >> can you verify that this is indeed the case?

> > > No, it doesn't work because that's the wrong set of options for gzip. > This works, however: > > gzip -dc Math-SymbolicX-NoSimplification-0.01.tar.gz |tar -xvf - > > Without the 'c' option, it operates directly on file and sends nothing > to tar.

Right, that wasn't the question.. as you can see this was some pseudo code.. the question was if your win32 can handle piped commands or not... For the heck of it, blow away the directories of the modules you had trouble with, refresh the sourcefiles and try again.. does the problem persist? -- Jos Boumans "If superman is so smart, why does he wear underpants over his trousers?" CPANPLUS http://cpanplus.sf.net

Sat Jan 22 10:02:46 2005 Guest - Correspondence added

From:

rrwo [...] cpan.org

[kane@dwim.org - Sat Jan 22 09:23:37 2005]: Show quoted text

> > There is no permissions problem in the .cpanplus directories. I can > > manually go there and extract archives, run make etc. with no > > problems.

Show quoted text

> ok, then why is there a permission denied coming from the extraction > process? there is /something/ lacking permissions /somehwere/

Not necessarily. It's common for programs to just call it a permissions problem when they can't access a file. It's particularly common when there's a directory that's been created where a utility wants to create one with the same name or write a file with that name. Suppose the archive is extracted, but somehow it thinks something went wrong and tried to re-extract it. But it can't, because the directory is already there, so it says there's a permissions problem. I think the problem may be with Archive::Extract or Archive::Tar. Show quoted text

> Right, that wasn't the question.. as you can see this was some pseudo > code.. the question was if your win32 can handle piped commands or > not...

Yes, it can. Show quoted text

> For the heck of it, blow away the directories of the modules you had > trouble with, refresh the sourcefiles and try again.. does the problem > persist?

Yes. I removed the 'id' directory and got the following: C:\TEMP>cpanp t Locale-Maketext-TieHash-nbsp-0.02 [MSG] Checking if source files are up to date [MSG] Retrieving C:\Documents and Settings\Rob.MAGNETO-XP\Application Data\.cpanplus\sourcefiles.2.13.stored Testing Locale::Maketext::TieHash::nbsp [MSG] Trying to get 'ftp://cpan.etla.org/pub/CPAN/authors/id/S/ST/STEFFENW/Local e-Maketext-TieHash-nbsp-0.02.tar.gz' [MSG] Trying to get 'ftp://cpan.etla.org/pub/CPAN/authors/id/S/ST/STEFFENW/CHECK SUMS' [MSG] Checksum matches for 'Locale-Maketext-TieHash-nbsp-0.02.tar.gz' Could not open file 'C:\Documents and Settings\Rob.MAGNETO-XP\Application Data\.cpanplus\5.8.6\build\Locale-Maketext-TieHash-nbsp-0.02\manifest': Permission denied at C:/Perl/site/lib/CPANPLUS/Internals/Extract.pm line 158 Could not extract 'Locale-Maketext-TieHash-nbsp-0.02/manifest' at C:/Perl/site/lib/CPANPLUS/Internals/Extract.pm line 158 Unable to extract 'C:\Documents and Settings\Rob.MAGNETO-XP\Application Data\.cpanplus\authors\id\S\ST\STEFFENW\Locale-Maketext-TieHash-nbsp-0.02.tar.gz': Could not extract 'Locale-Maketext-TieHash-nbsp-0.02/manifest' at C:/Perl/site/lib/CPANPLUS/Module.pm line 469 gzip.EXE: stdout: Invalid argument C:\Perl\bin\tar.EXE: Cannot open -: No such file or directory C:\Perl\bin\tar.EXE: Error is not recoverable: exiting now

Sat Jan 22 10:09:36 2005 kane [...] dwim.org - Correspondence added

From:	"Jos I. Boumans" <kane [...] dwim.org>
Subject:	Re: [cpan #11139] Calls to tar/gzip lock up on Windows
Date:	Sat, 22 Jan 2005 16:05:22 +0100
To:	bug-CPANPLUS [...] rt.cpan.org
RT-Send-Cc:

On Jan 22, 2005, at 4:02 PM, Guest via RT wrote: Show quoted text

> > I think the problem may be with Archive::Extract or Archive::Tar.

possibly.. Show quoted text

>> Right, that wasn't the question.. as you can see this was some pseudo >> code.. the question was if your win32 can handle piped commands or >> not...

> > Yes, it can.

That's good to hear, then i wonder though why the piped opens are causing the program to hang... Show quoted text

>> For the heck of it, blow away the directories of the modules you had >> trouble with, refresh the sourcefiles and try again.. does the problem >> persist?

> > Yes. I removed the 'id' directory and got the following:

ehm, the /extraction/ location should be removed (as that is where we got the errors, not from the original tarball directory). So, with a default setup, that'd be the 'build' directory... -- Jos Boumans How do I prove I am not crazy to people who are? CPANPLUS http://cpanplus.sf.net

Sat Jan 22 10:49:22 2005 Guest - Correspondence added

From:

rrwo [...] cpan.org

[kane@dwim.org - Sat Jan 22 10:09:36 2005]: Show quoted text

> That's good to hear, then i wonder though why the piped opens are > causing the program to hang...

I think Windows has a problem with Perl alarm timeouts for blocking file reads. This problem came up when I was doing some testing on Time::Out. Show quoted text

> >> For the heck of it, blow away the directories of the modules you had > >> trouble with, refresh the sourcefiles and try again.. does the problem > >> persist?

> > > > Yes. I removed the 'id' directory and got the following:

> ehm, the /extraction/ location should be removed (as that is where we > got the errors, not from the original tarball directory). So, with a > default setup, that'd be the 'build' directory...

Ok, that fixed it. So what's happenng in that directory that is causing this problem?

Sat Jan 22 10:54:28 2005 kane [...] dwim.org - Correspondence added

From:	"Jos I. Boumans" <kane [...] dwim.org>
Subject:	Re: [cpan #11139] Calls to tar/gzip lock up on Windows
Date:	Sat, 22 Jan 2005 16:50:18 +0100
To:	bug-CPANPLUS [...] rt.cpan.org
RT-Send-Cc:

On Jan 22, 2005, at 4:49 PM, Guest via RT wrote: Show quoted text

> [kane@dwim.org - Sat Jan 22 10:09:36 2005]: >

>> That's good to hear, then i wonder though why the piped opens are >> causing the program to hang...

> > I think Windows has a problem with Perl alarm timeouts for blocking > file > reads. This problem came up when I was doing some testing on Time::Out.

Hmm... we're not using any alarm directly, but the underlying code (ipc::run) might be, not sure... Any suggestions are of course welcome... Show quoted text

>>>> For the heck of it, blow away the directories of the modules you had >>>> trouble with, refresh the sourcefiles and try again.. does the >>>> problem >>>> persist?

>>> >>> Yes. I removed the 'id' directory and got the following:

>> ehm, the /extraction/ location should be removed (as that is where we >> got the errors, not from the original tarball directory). So, with a >> default setup, that'd be the 'build' directory...

> > Ok, that fixed it. > > So what's happenng in that directory that is causing this problem?

Then it is a permissions problem after all -- the permissions of the files in the tarball are apparently set to +r only (at least the ones with errors), causing archive::tar to generate an error, and archive::extract to use option #2 to extract files, namely /bin/gzip and /bin/tar.. and they hang. Now not having this error occur is a bit tricky, but is possibly avoidable -- i'll look into it. But how to make gzip and tar not hang i'm not sure about... any help there would be much appreciated. -- Jos Boumans "Never ask a man what computer he uses. If it's a Mac, he'll tell you. If it's not, why embarrass him?" - Tom Clancy CPANPLUS http://cpanplus.sf.net

Sun Jan 23 11:14:09 2005 kane [...] cpan.org - Status changed from 'new' to 'stalled'

Thu Feb 17 08:01:25 2005 kane [...] cpan.org - Correspondence added

[kane@dwim.org - Sat Jan 22 10:54:28 2005]: Show quoted text

> >>> Yes. I removed the 'id' directory and got the following:

> >> ehm, the /extraction/ location should be removed (as that is where we > >> got the errors, not from the original tarball directory). So, with a > >> default setup, that'd be the 'build' directory...

> > > > Ok, that fixed it. > > > > So what's happenng in that directory that is causing this problem?

> Then it is a permissions problem after all -- the permissions of the > files in the tarball are apparently set to +r only (at least the ones > with errors), causing archive::tar to generate an error, and > archive::extract to use option #2 to extract files, namely /bin/gzip > and /bin/tar.. and they hang.

As a sort of 'safety net' CPANPLUS now does a chmod -R +w on the extract dir, in case we find naughty files that are only +r so a re-extract will work without causing errors. This change is available from CPANPLUS 0.053 and up. Show quoted text

> Now not having this error occur is a bit tricky, but is possibly > avoidable -- i'll look into it. But how to make gzip and tar not hang > i'm not sure about... any help there would be much appreciated.

This issue i don't know how to resolve, but is most likely caused either by the IPC::* module loaed by IPC::Cmd (probably IPC::Run since you are usually on win32) and/or the OS. Investigating there might be worth while, but is no longer a CPANPLUS issue, so i'll close this bug.

Thu Feb 17 08:01:26 2005 kane [...] cpan.org - Status changed from 'stalled' to 'resolved'

Thu Feb 17 08:01:26 2005 The RT System itself - Status changed from 'stalled' to 'open'