Bug #130120 for ExtUtils-Install: ExtUtils-Install/t/Installed.t: intermittent crashes in Perl 5 test suite

On Sat Jul 20 09:33:39 2019, JKEENAN wrote: Show quoted text
Here is additional evidence of intermittent failures in cpan/ExtUtils-Install/t/Installed.t in the Perl 5 core distribution test suite. Several times each month I smoke-test blead (or, occasionally, other branches) on FreeBSD-11 with a configuration that exercises 3 different C-compilers -- gcc-7.3.0, g++-7.3.0, clang-6.0.1 -- with 6 different combinations of Configure switches for each. See, for example, http://perl5.test-smoke.org/report/109163. On 5 occasions within the past 90 days, my logs/smokecurrent/log*.log file has recorded warnings like this: ##### Can't cd to (/usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/auto/) ExtTest: No such file or directory at /usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/ExtUtils/Installed.pm line 201. ##### Usually the unit test runs to completion and Installed.t is graded PASS -- as is the smoke-test run as a whole. And usually this warning appears only once in the 18 different configurations in the run. However, in the smoke-test I ran yesterday (the URL above), I got the warning 3 times, one of which resulted in a FAIL for Installed.t and hence an overall grade of FAIL for the smoke-test run. Here are the 3 instances, each preceded by the configuration in effect at the time. ##### 178: [2020-03-13 17:13:38+0000] Configuration: -Dusedevel -Duseithreads -Doptimize="-O2 -pipe -fstack-protector -fno-strict-aliasing" -Dcc="clang -Qunused-arguments" -Duselongdouble -DDEBUGGING [2020-03-13 17:13:38+0000] ------------------------------------------------------------------------------ 206: Can't cd to (/usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/auto/) ExtTest: No such file or directory at /usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/ExtUtils/Installed.pm line 201. 1811: [2020-03-13 21:04:06+0000] Configuration: -Dusedevel -Duseithreads -Doptimize="-O2 -pipe -fstack-protector -fno-strict-aliasing" -Dcc="gcc" -Duselongdouble -DDEBUGGING [2020-03-13 21:04:06+0000] ------------------------------------------------------------------------------ 1980: Can't cd to ../../../.. from ./t/n4k6w86Mau/Big-Dummy/install-test/other_lib/perl/Big: No such file or directory at /usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/File/Find.pm line 470. cannot fetch initial working directory: No such file or directory at t/Installed.t line 83. # Looks like your test exited with 2 just after 18. # Will use Digest::MD5 error while running harness target 'test_harness': 1 at /usr/home/jkeenan/p5smoke/lib/Test/Smoke/App/RunSmoke.pm line 118. 2197: [2020-03-13 21:41:46+0000] Configuration: -Dusedevel -Duseithreads -Doptimize="-O2 -pipe -fstack-protector -fno-strict-aliasing" -Dcc="gcc" -Dusemorebits -DDEBUGGING [2020-03-13 21:41:46+0000] ------------------------------------------------------------------------------ 2366: Can't cd to (./t/) y69u52MVuL: No such file or directory at /usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/ExtUtils/Installed.pm line 201. ##### The failures occur with different C-compilers and different Configure switches. The 3 instances yesterday were each DEBUGGING builds, but the number of instances is too small to attribute statistical significance to that. As previously reported, the test failure (and presumed segfault) happened after test 18 in Installed.t. ##### $ cd t;./perl harness -v ../cpan/ExtUtils-Install/t/Installed.t; cd - ok 1 - use ExtUtils::Installed; ok 2 - _is_prefix() should match valid path prefix ok 3 - ... should not match wrong prefix ok 4 - _is_type() should be true for type of "all" ok 5 - ... should find doc file in man1dir ok 6 - ... but not prog file in man1dir ok 7 - ... should find doc file in man3dir ok 8 - ... but not prog file in man3dir ok 9 - ... should find prog file under /usr/local ok 10 - ... should not find doc file outside path ok 11 - ... nor prog file outside path ok 12 - ... nor other type anywhere ok 13 - _is_under() should return true with no dirs ok 14 - ... should find no file not under dirs ok 15 - ... should find file under dir ok 16 ok 17 ok 18 ok 19 - An object of class 'ExtUtils::Installed' isa 'ExtUtils::Installed' ##### Here is the relevant section on Installed.t. Note "line 83". ##### rmtree 'auto/FakeMod'; ok( mkpath('auto/FakeMod') ); END { rmtree 'auto' } ### <-- line 83 - reported failure ok(open(PACKLIST, '>auto/FakeMod/.packlist')); print PACKLIST 'list'; close PACKLIST; ok(open(FAKEMOD, '>auto/FakeMod/FakeMod.pm')); print FAKEMOD <<'FAKE'; package FakeMod; use vars qw( $VERSION ); $VERSION = '1.1.1'; 1; FAKE close FAKEMOD; my $fake_mod_dir = File::Spec->catdir(cwd(), 'auto', 'FakeMod'); { # avoid warning and death by localizing glob local *ExtUtils::Installed::Config; %ExtUtils::Installed::Config = ( %Config, archlibexp => cwd(), sitearchexp => $fake_mod_dir, ); # should find $fake_mod_dir via '.' in @INC local @INC = @INC; push @INC, '.' if not $INC[-1] eq '.'; my $realei = ExtUtils::Installed->new(); isa_ok( $realei, 'ExtUtils::Installed' ); isa_ok( $realei->{Perl}{packlist}, 'ExtUtils::Packlist' ); is( $realei->{Perl}{version}, $Config{version}, 'new() should set Perl version from %Config' ); ok( exists $realei->{FakeMod}, 'new() should find modules with .packlists'); isa_ok( $realei->{FakeMod}{packlist}, 'ExtUtils::Packlist' ); is( $realei->{FakeMod}{version}, '1.1.1', '... should find version in modules' ); } ##### This gives us a clue to the source of the problem. Each of the test runs that either warned or errored was a threaded build. File::Path::rmtree() is documented to have problems with threaded builds. From the documentation for version 2.16: ##### File::Path "rmtree" and "remove_tree" will not work with multithreaded applications due to its use of "chdir". At this time, no warning or error is generated in this situation. You will certainly encounter unexpected results. The implementation that surfaces this limitation will not be changed. ##### Can Installed.t be adapted to work consistently on threaded builds? Thank you very much. Jim Keenan

On Sat Mar 14 09:16:55 2020, JKEENAN wrote: [snip] Show quoted text
Further weirdness: The string 'ExtTest' is not found in the ExtUtils-Install distribution. The only place where it occurs in the Perl 5 source code is in this test file in a *different* cpan/ distro: ##### [perl] 553 $ ack ExtTest . cpan/ExtUtils-Constant/t/Constant.t 50:my $package = "ExtTest"; 157: # Writing Makefile for ExtTest 285: # Need to make distclean to remove ../../lib/ExtTest.pm 660:} elsif ($@ !~ /Your vendor has not defined ExtTest macro NOTDEF/) { 668:my $notthere = eval { &ExtTest::NOTTHERE; }; 671:} elsif ($@ !~ /NOTTHERE is not a valid ExtTest macro/) { 707:$notdef = eval { &ExtTest::So }; 710:} elsif ($@ !~ /^So is not a valid ExtTest macro/) { 718:$notdef = eval { &ExtTest::EW }; 721:} elsif ($@ !~ /^EW is not a valid ExtTest macro/) { 835:ExtTest.xs: In function `constant_1': 836:ExtTest.xs:80: warning: multi-character character constant 837:ExtTest.xs:80: warning: case value out of range ##### jimk