Skip Menu |

This queue is for tickets about the Test-Harness CPAN distribution.

Report information
The Basics
Id: 58428
Status: resolved
Priority: 0/
Queue: Test-Harness

People
Owner: andy [...] hexten.net
Requestors: djmitche [...] gmail.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Test::Harness uses sh -c to run tests, not in PATH
Date: Tue, 15 Jun 2010 17:54:13 -0500
To: bug-test-harness [...] rt.cpan.org
From: "Dustin J. Mitchell" <djmitche [...] gmail.com>
As part of Amanda, we run tests that are not in a subdirectory, e.g., /usr/bin/perl -I. -I. -e 'use Test::Harness qw(&runtests); runtests(sort @ARGV);' Amanda_Archive Amanda_Config Amanda_ClientService Amanda_Debug (there are more tests, but you get the idea) The problem is, in Test::Harness 2.30, these tests are run with sh -c $testname which of course doesn't work because '.' is not in PATH and the paths are not qualified. I can think of a few backward-compatible ways to fix this, e.g., runtests(sort map { "./$_" } @ARGV), but it seems like a pretty serious regression that should either be fixed or documented. A google search for the error finds a number of CPAN testers failures: http://www.google.com/search?hl=en&client=safari&rls=en&q=%22open3%3A+exec+of%22+%22tap%3A%3Aparser%3A%3Aiterator%3A%3Aprocess%22&aq=f&aqi=&aql=&oq=&gs_rfai= and not much else. Dustin
Subject: Re: [rt.cpan.org #58428] Test::Harness uses sh -c to run tests, not in PATH
Date: Tue, 15 Jun 2010 22:12:02 -0700
To: bug-Test-Harness [...] rt.cpan.org
From: Michael G Schwern <schwern [...] pobox.com>
On 2010.6.15 3:54 PM, Dustin J. Mitchell via RT wrote: Show quoted text
> As part of Amanda, we run tests that are not in a subdirectory, e.g., > > /usr/bin/perl -I. -I. -e 'use Test::Harness qw(&runtests); > runtests(sort @ARGV);' Amanda_Archive Amanda_Config > Amanda_ClientService Amanda_Debug > > (there are more tests, but you get the idea) > > The problem is, in Test::Harness 2.30, these tests are run with > sh -c $testname > which of course doesn't work because '.' is not in PATH and the paths > are not qualified.
Thanks for your report. I'm pretty sure Test::Harness never actually calls "sh -c", at least not directly. So something weird is going on. How did you get that impression? I can also say with some certainty that Test::Harness never assumed the test files would be in a subdirectory and it should work fine. Its possible open(FILE, "$cmd |") is broken in your Perl. I think what's missing here is a clear error message. Could you send... * The complete output showing the error * The complete output of "perl -V" Also the current version of Test::Harness is 3.21 and its significantly rearchitected. 2.30 is approaching seven years old. You probably want to upgrade just on general principles. It also indicates that you're using Perl 5.8.1 or 5.6.2, both seven years old. Show quoted text
> I can think of a few backward-compatible ways to fix this, e.g., > runtests(sort map { "./$_" } @ARGV), but it seems like a pretty > serious regression that should either be fixed or documented. A > google search for the error finds a number of CPAN testers failures: > > http://www.google.com/search?hl=en&client=safari&rls=en&q=%22open3%3A+exec+of%22+%22tap%3A%3Aparser%3A%3Aiterator%3A%3Aprocess%22&aq=f&aqi=&aql=&oq=&gs_rfai= > > and not much else.
What you're seeing there is open3 failing because of an enormous PERL5LIB environment variable. CPAN clients used to get a bit carried away. I don't think its relevant to this problem. -- Insulting our readers is part of our business model. http://somethingpositive.net/sp07122005.shtml
As Schwern says Test::Harness 2.30 is /very/ old. The code's been completely rewritten since then.
Subject: Re: [rt.cpan.org #58428] Test::Harness uses sh -c to run tests, not in PATH
Date: Wed, 16 Jun 2010 11:12:41 -0500
To: bug-Test-Harness [...] rt.cpan.org
From: "Dustin J. Mitchell" <djmitche [...] gmail.com>
On Wed, Jun 16, 2010 at 12:12 AM, Michael G Schwern via RT <bug-Test-Harness@rt.cpan.org> wrote: Show quoted text
> Thanks for your report.
We are starting to do automated tests on ubuntu-10.04, and this is the first place we've seen this failure (well, first two places - it happens on 32- and 64-bit systems). Since we want Amanda to work on systems "out of the box", upgrading or otherwise tweaking the distro install is not an option. Show quoted text
> * The complete output showing the error
I boiled this down quite a bit. test$ cat mytest use Test::More tests => 1; pass("yay"); test$ chmod -x mytest test$ perl mytest 1..1 ok 1 - yay test$ perl -e 'use Test::Harness qw(&runtests); runtests(sort @ARGV);' mytest mytest .. ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.00 usr 0.03 sys + 0.00 cusr 0.04 csys = 0.07 CPU) Result: PASS test$ chmod +x mytest test$ perl -e 'use Test::Harness qw(&runtests); runtests(sort @ARGV);' mytest mytest .. open3: exec of mytest failed at /usr/share/perl5/TAP/Parser/Iterator/Process.pm line 168 mytest .. Dubious, test returned 255 (wstat 65280, 0xff00) No subtests run Test Summary Report ------------------- mytest (Wstat: 65280 Tests: 0 Failed: 0) Non-zero exit status: 255 Parse errors: No plan found in TAP output Files=1, Tests=0, 0 wallclock secs ( 0.01 usr 0.02 sys + 0.00 cusr 0.02 csys = 0.05 CPU) Result: FAIL Failed 1/1 test programs. 0/0 subtests failed. So it looks like something in the test infrastructure is deciding to run the script if its executable flag is set, although it's using execvpe in this case, which isn't right. I can fix this in the Amanda Makefiles with chmod -x, but I still think there's a bug here - either in a lack of documentation, or in an unintended behavior change. Show quoted text
> * The complete output of "perl -V"
Summary of my perl5 (revision 5 version 10 subversion 1) configuration: Platform: osname=linux, osvers=2.6.24-27-server, archname=i486-linux-gnu-thread-multi uname='linux vernadsky 2.6.24-27-server #1 smp fri mar 12 01:45:06 utc 2010 i686 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.10.1 -Dsitearch=/usr/local/lib/perl/5.10.1 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.10.1 -Dd_dosuid -des' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=undef, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2 -g', cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.4.3', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/lib64 libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=/lib/libc-2.11.1.so, so=so, useshrplib=true, libperl=libperl.so.5.10.1 gnulibc_version='2.11.1' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_REENTRANT_API Built under linux Compiled at Apr 23 2010 07:36:53 @INC: /etc/perl /usr/local/lib/perl/5.10.1 /usr/local/share/perl/5.10.1 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl Show quoted text
> Also the current version of Test::Harness is 3.21 and its significantly > rearchitected.  2.30 is approaching seven years old.  You probably want to > upgrade just on general principles.  It also indicates that you're using Perl > 5.8.1 or 5.6.2, both seven years old.
Sorry - I typo'd the version! root@buildbot-ub1004-32:~# perl -MTest::Harness -e 'print "$Test::Harness::VERSION\n"' 3.20 Show quoted text
> I'm pretty sure Test::Harness never actually calls "sh -c", at least not > directly.  So something weird is going on.  How did you get that impression?
We have a test named =setupcache - intended to get that name to sort first in the list of tests. Apparently when that file is executable, it gets run by the shell: test$ perl -e 'use Test::Harness qw(&runtests); runtests(sort @ARGV);' =mytest =mytest .. sh: =mytest: not found =mytest .. Dubious, test returned 127 (wstat 32512, 0x7f00) No subtests run Making it non-executable fixes the problem. Dustin P.S. Test::Harness works fine for Amanda on perl-5.6.2 - we test it every commit :)
On Wed Jun 16 12:12:51 2010, djmitche@gmail.com wrote: Show quoted text
> We are starting to do automated tests on ubuntu-10.04, and this is the > first place we've seen this failure (well, first two places - it > happens on 32- and 64-bit systems). Since we want Amanda to work on > systems "out of the box", upgrading or otherwise tweaking the distro > install is not an option.
Ubuntu already packages a recent version (3.20 right now) of Test::Harness as libtest- harness-perl. Could you install that? Since the 2.xx line of Test::Harness isn't developed any more and - in any event - you're not able to do any upgrades I don't really see what we can do for you... Presumably you couldn't installed a fixed version anyway?
From: djmitche [...] gmail.com
On Wed Jun 16 12:26:02 2010, ANDYA wrote: Show quoted text
> Ubuntu already packages a recent version (3.20 right now) of > Test::Harness as libtest- > harness-perl. Could you install that?
That's what's installed. Show quoted text
> Since the 2.xx line of Test::Harness isn't developed any more and - in > any event - you're not > able to do any upgrades I don't really see what we can do for you... > Presumably you couldn't > installed a fixed version anyway?
I'm sorry if I wasn't clear in my last post. 2.30 was a typo. I should have written 3.20.
Subject: Re: [rt.cpan.org #58428] Test::Harness uses sh -c to run tests, not in PATH
Date: Wed, 16 Jun 2010 10:48:35 -0700
To: bug-Test-Harness [...] rt.cpan.org
From: Michael G Schwern <schwern [...] pobox.com>
On 2010.6.16 9:12 AM, Dustin J. Mitchell via RT wrote: Show quoted text
>> * The complete output showing the error
> > I boiled this down quite a bit.
Thanks, I can replicate this on OS X with 3.18 through 3.21. Looks like a bug introduced in 3.18. Show quoted text
> test$ cat mytest > use Test::More tests => 1; > pass("yay"); > test$ chmod -x mytest > test$ perl mytest > 1..1 > ok 1 - yay > test$ perl -e 'use Test::Harness qw(&runtests); runtests(sort @ARGV);' mytest > mytest .. ok > All tests successful. > Files=1, Tests=1, 0 wallclock secs ( 0.00 usr 0.03 sys + 0.00 cusr > 0.04 csys = 0.07 CPU) > Result: PASS > test$ chmod +x mytest > test$ perl -e 'use Test::Harness qw(&runtests); runtests(sort @ARGV);' mytest > mytest .. open3: exec of mytest failed at > /usr/share/perl5/TAP/Parser/Iterator/Process.pm line 168 > mytest .. Dubious, test returned 255 (wstat 65280, 0xff00) > No subtests run > > Test Summary Report > ------------------- > mytest (Wstat: 65280 Tests: 0 Failed: 0) > Non-zero exit status: 255 > Parse errors: No plan found in TAP output > Files=1, Tests=0, 0 wallclock secs ( 0.01 usr 0.02 sys + 0.00 cusr > 0.02 csys = 0.05 CPU) > Result: FAIL > Failed 1/1 test programs. 0/0 subtests failed.
-- Who invented the eponym?
This should be fixed now.