Bug #70232 for ExtUtils-MakeMaker: Hang in t/INSTALL

Fri Aug 12 09:47:16 2011 ucjddavis [...] gmail.com - Ticket created

ExtUtils-MakeMaker is not completing t/INSTALL_BASE.t runs during package installations when it is called. I am encountering this with every new FreeBSD server that I am building and using CPAN to install modules. ## system info: 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 ## Perl Info: 5.12.4 standard freebsd port installation ## sample output the out put here is from the installation of YAML as the first package after running CPAN for the first time. Manifying blib/man3/ExtUtils::MM_Any.3 MSTROUT/ExtUtils-MakeMaker-6.59.tar.gz /usr/bin/make -- OK Running make test PERL_DL_NONLAZY=1 /usr/local/bin/perl "-Iblib/arch" "-Iblib/lib" "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/00compile.t ............. ok t/arch_check.t ............ ok t/backwards.t ............. ok t/basic.t ................. ok t/build_man.t ............. ok t/cd.t .................... ok t/config.t ................ ok t/dir_target.t ............ ok t/FIRST_MAKEFILE.t ........ ok t/fix_libs.t .............. ok t/fixin.t ................. ok t/hints.t ................. ok t/INST.t .................. ok t/INST_PREFIX.t ........... ok t/INSTALL_BASE.t .......... 1/20 The system hangs here and will proceed no further. If you ^c out of this the install may proceed with seeming success or it may fail depending on the package This does not occur with ExUtils-MakeMaker-6.56 on the same system. Any corrections to this issue or methods to work around it would be great.

Sun Sep 25 01:11:42 2011 mschwern [...] cpan.org - Subject changed from (no value) to 'Hang in t/INSTALL_BASE.t on FreeBSD'

Sun Sep 25 01:11:42 2011 mschwern [...] cpan.org - Severity Critical changed to Normal

Wed Oct 05 16:44:55 2011 JPEREGR [...] cpan.org - Ticket #71480: Ticket created

Subject:

hangs during make test INSTALL_BASE

[peregrin@bert ExtUtils-MakeMaker-6.59]$ make test PERL_DL_NONLAZY=1 /usr/bin/perl "-Iblib/arch" "-Iblib/lib" "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/00compile.t ............. ok t/arch_check.t ............ ok t/backwards.t ............. ok t/basic.t ................. ok t/build_man.t ............. ok t/cd.t .................... ok t/config.t ................ ok t/dir_target.t ............ ok t/FIRST_MAKEFILE.t ........ ok t/fix_libs.t .............. ok t/fixin.t ................. ok t/hints.t ................. ok t/INST.t .................. ok t/INST_PREFIX.t ........... ok t/INSTALL_BASE.t .......... 1/20 ^C sits at 1/20 until broken out with a ^C. Happens in 6.59 and 6.61_01. Also reported in bugreport #70232 and http://www.perlmonks.org/bare/index.pl? node_id=918565 OS: FreeBSD 7.4 Perl: 5.12.4 (system perl via ports)

Thu Oct 06 00:08:57 2011 mschwern [...] cpan.org - Ticket #71480: Merged into ticket #70232

Thu Oct 06 00:08:57 2011 mschwern [...] cpan.org - Merged into ticket #70232

Thu Oct 06 00:35:15 2011 mschwern [...] cpan.org - Correspondence added

Lacking a FreeBSD machine, or knowledge of FreeBSD's quirks, I'm going to have to rely on you to do some investigation. I order to fix this, I need one of you to do the following... 1. Download a fresh copy of the ExtUtils-MakeMaker 6.61_01 tarball from CPAN. 2. Unpack it. 3. Run "perl Makefile.PL" and send on the output. 4. Run this command and send the last screen full of output when it hangs. PERLDB_OPTS='NonStop AutoTrace' perl -Ilib -dw t/INSTALL_BASE.t 5. Run this similar command and send me the contents of trace.out. PERLDB_OPTS='NonStop AutoTrace LineInfo=tperl.out' perl -Ilib -dw t/INSTALL_BASE.t You may have to kill the program once it hangs. Those last two will give complete traces of what's going on in the test hopefully revealing the bug. If you're knowledgeable with the debugger, you can remove NonStop and try to debug it yourself. That would be most appreciated. Let me check other things that might be interfering. 1. Please verify that you have the same problem when installing from a fresh tarball downloaded directly from CPAN (ie. not from the FreeBSD ports collection) and the install is done manually (ie. not from in a CPAN shell). 2. Please verify that you are not using a jail or other security system. If you are, lmk. 3. [Optional] Please download and compile 5.12.4 from source (ie. not from FreeBSD) and see if the problem persists. 4. What version of ExtUtils::Install and ExtUtils::Packlist do you have installed?

Thu Oct 06 00:35:16 2011 The RT System itself - Status changed from 'new' to 'open'

Thu Oct 06 17:11:37 2011 james_peregrino [...] harvard.edu - Correspondence added

CC:	JPEREGR [...] cpan.org, ucjddavis [...] gmail.com
Subject:	Re: [rt.cpan.org #70232] Hang in t/INSTALL_BASE.t on FreeBSD
Date:	Thu, 6 Oct 2011 17:11:06 -0400
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	James Peregrino <james_peregrino [...] harvard.edu>

On Oct 6, 2011, at 12:35 AM, bug-ExtUtils-MakeMaker@rt.cpan.org wrote: Show quoted text

> Lacking a FreeBSD machine, or knowledge of FreeBSD's quirks, I'm going > to have to rely on you to do some investigation. > > I order to fix this, I need one of you to do the following... > > 1. Download a fresh copy of the ExtUtils-MakeMaker 6.61_01 tarball from > CPAN. > 2. Unpack it. > 3. Run "perl Makefile.PL" and send on the output.

see attached file "makefileplscript.txt Show quoted text

> 4. Run this command and send the last screen full of output when it hangs.. > > PERLDB_OPTS='NonStop AutoTrace' perl -Ilib -dw t/INSTALL_BASE.t

see attached file perldbopts-lastscreen.txt Show quoted text

> > 5. Run this similar command and send me the contents of trace.out. > > PERLDB_OPTS='NonStop AutoTrace LineInfo=tperl.out' perl -Ilib -dw > t/INSTALL_BASE.t >

see attached file tperl.out Show quoted text

> [snip] > Let me check other things that might be interfering. > > 1. Please verify that you have the same problem when installing from a > fresh tarball downloaded directly from CPAN (ie. not from the FreeBSD > ports collection) and the install is done manually (ie. not from in a > CPAN shell).

same problem Show quoted text

> > 2. Please verify that you are not using a jail or other security system. > If you are, lmk. >

nope, no jails or other security. Show quoted text

> 3. [Optional] Please download and compile 5.12.4 from source (ie. not > from FreeBSD) and see if the problem persists.

used perlbrew to install a virgin 5.12.4 -> no problems, ExtUtils::MakeMaker passes all tests Show quoted text

> > 4. What version of ExtUtils::Install and ExtUtils::Packlist do you have > installed?

System Perl (via ports) - ExtUtils::Install 1.54, ExtUtils::PackList 1.43 - these are installed as a separate port and are the latest version in the ports system. Show quoted text

>

Message body is not shown because sender requested not to inline it.

Download tperl.out
application/octet-stream 3.1m

Message body not shown because it is not plain text.

-- -- James Peregrino - http://people.dce.harvard.edu/~peregrin/business-card.html

Sat Oct 08 19:19:44 2011 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #70232] Hang in t/INSTALL_BASE.t on FreeBSD
Date:	Sat, 08 Oct 2011 16:19:28 -0700
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

On 2011.10.6 2:11 PM, james_peregrino@harvard.edu via RT wrote: Show quoted text

>> 1. Download a fresh copy of the ExtUtils-MakeMaker 6.61_01 tarball from >> CPAN. >> 2. Unpack it. >> 3. Run "perl Makefile.PL" and send on the output.

> > see attached file "makefileplscript.txt

Ok, that tells me your modules are basically up to date. Thanks. Show quoted text

>> 5. Run this similar command and send me the contents of trace.out. >> >> PERLDB_OPTS='NonStop AutoTrace LineInfo=tperl.out' perl -Ilib -dw >> t/INSTALL_BASE.t

If that trace is correct, it appears to be hanging running "make install 2>&1" on line 45: my $install_out = run("$make install"); Please try some things to figure out what's causing the hang. Replace line 45 with each of these in turn and tell me if it hangs or not. my $install_out = `$make install 2>&1`; my $install_out = `$make install`; my $install_out; system("$make install 2>&1"); my $install_out; system("$make install"); Those last two will cause the next two tests to fail. Send on the output of each run please. If the system() calls hang they should tell us where. Show quoted text

>> 3. [Optional] Please download and compile 5.12.4 from source (ie. not >> from FreeBSD) and see if the problem persists.

> > used perlbrew to install a virgin 5.12.4 -> no problems, ExtUtils::MakeMaker > passes all tests

Thank you. Now we know it's something FreeBSD did. The question is what. Show quoted text

>> 4. What version of ExtUtils::Install and ExtUtils::Packlist do you have >> installed?

> > System Perl (via ports) - ExtUtils::Install 1.54, ExtUtils::PackList 1.43 > - these are installed as a separate port and are the latest version in the > ports system.

Hmm. The real 5.12.4 is using ExtUtils::Install 1.55 and ExtUtils::PackList 1.44, but they don't contain any significant changes. I don't see any changes to the CPAN module in the port. Just to be sure, could you diff the ports supplied ExtUtils::Install and ExtUtils::PackList with the 1.54 on CPAN please? -- Stabbing you in the face for your own good.

Tue Oct 25 13:50:14 2011 RWSTAUNER [...] cpan.org - Correspondence added

RT-Send-CC:

schwern [...] pobox.com

trying FreeBSD 8.2 in a virtualbox vm I noticed this issue as well. switching to system("$make install 2>&1") displays this additional output: Installing ../dummy-install/lib/perl5/Big/Dummy.pm Installing ../dummy-install/lib/perl5/Big/Liar.pm Installing ../dummy-install/man/man1/program.1 Installing ../dummy-install/man/man3/Big::Dummy.pm Installing ../dummy-install/bin/program FreeBSD: Registering installation in the package database FreeBSD: Cannot determine module description Copying the 'AUTHOR => '... line from Big/Dummy's Makefile to Big/Liar's (in t/lib/MakeMaker/Setup/BFD.pm) seems to allow the system command to proceed, though of course then lots of other things break.

Tue Oct 25 15:41:22 2011 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #70232] Hang in t/INSTALL_BASE.t on FreeBSD
Date:	Tue, 25 Oct 2011 12:41:06 -0700
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

On 2011.10.25 10:50 AM, Randy Stauner via RT wrote: Show quoted text

> trying FreeBSD 8.2 in a virtualbox vm I noticed this issue as well. > > switching to system("$make install 2>&1") displays this additional output: > > Installing ../dummy-install/lib/perl5/Big/Dummy.pm > Installing ../dummy-install/lib/perl5/Big/Liar.pm > Installing ../dummy-install/man/man1/program.1 > Installing ../dummy-install/man/man3/Big::Dummy.pm > Installing ../dummy-install/bin/program > FreeBSD: Registering installation in the package database > FreeBSD: Cannot determine module description

Whoa. That last bit is NOT from MakeMaker. I think you're using some FreeBSD patched version of ExtUtils::Install. Could you please send me the contents of your ExtUtils::Install? "perldoc -l ExtUtils::Install" should find it. Show quoted text

> Copying the 'AUTHOR => '... line from Big/Dummy's Makefile to Big/Liar's > (in t/lib/MakeMaker/Setup/BFD.pm) seems to allow the system command to > proceed, though of course then lots of other things break.

That's... really strange. I wonder if whatever they're doing cannot handle a module without an author. -- 184. When operating a military vehicle I may *not* attempt something "I saw in a cartoon". -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army http://skippyslist.com/list/

Tue Oct 25 18:24:23 2011 mschwern [...] cpan.org - Correspondence added

Ok, I installed a FreeBSD 8.2 VM and figured out what's going on. Long story short, the problem is likely upstream. BSD is injecting code into ExtUtils::MM_Unix, MakeMaker and Packlist which is likely interfering with the install. It should be reported to FreeBSD. The problem is BSDPAN::ExtUtils::Packlist::get_dir_list(). It's sneakily loaded by BSDPAN::Override which appears to be sneakily loaded by BSDPAN's own Config.pm which is sneakily loaded when you "use Config". BSDPAN::EU::Packlist adds code to register the module in the package database. get_dir_list() is shaving entries off file paths using a regex instead of File::Spec or File::Basename inside a loop. The result is a path like "../dummy-install" as used in the test causes an infinite loop. It attempts to guard against this but fails. It would be much safer to just split the whole path and iterate through it. There is a work around. The special BSD behavior can be turned off by flipping some environment variables. PORTOBJFORMAT seems most benign. I'll have the tests run with that when installing.

Tue Oct 25 19:02:25 2011 mschwern [...] cpan.org - Correspondence added

Okie doke, that should do it. "make test" now passes on FreeBSD 8.2. https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/commit/eb2dae36fc298ca2a956b829c6256a1f4e1fded1 I've made that report about BSDPAN upstream. It's supposed to be at http://www.freebsd.org/cgi/query-pr.cgi?pr=162016 but it's not yet.

Tue Oct 25 19:02:26 2011 mschwern [...] cpan.org - Status changed from 'open' to 'resolved'

Bug #70232 for ExtUtils-MakeMaker: Hang in t/INSTALL_BASE.t on FreeBSD