| Subject: | t/pod.t: Test fails to detect non-installation of test helper module |
| Date: | Sat, 27 Jun 2015 09:57:51 -0400 |
| To: | bug-File-Path [...] rt.cpan.org |
| From: | James E Keenan <jkeen [...] verizon.net> |
The way t/pod.t detects the presence or absence of Pod::Test and
Pod::Test:::Coverage appears to be defective.
(Editorial comment: I'm not a big fan of either of these modules. As a
module author, I'm sufficiently picky about running 'podchecker' to not
want to bother cluttering up my distributions with POD tests. I'm
particularly opposed to Pod::Test::Coverage because it presumes there is
only one way to structure your documentation and fails you if you don't
adhere to that structure. So I'll use them here only because we're
maintaining existing code.)
If I have Pod::Test installed but do *not* have Pod::Test::Coverage
installed, I get this output:
#####
$ PERL_AUTHOR_TESTING=1 && export PERL_AUTHOR_TESTING
$ perl Makefile.PL
$ make
$ prove -vb t/pod.t
t/pod.t ..
1..2
ok 1 - POD test for lib/File/Path.pm
Undefined subroutine &main::pod_coverage_ok called at t/pod.t line 53.
# Looks like you planned 2 tests but ran 1.
# Looks like your test exited with 255 just after 1.
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 1/2 subtests
Test Summary Report
-------------------
t/pod.t (Wstat: 65280 Tests: 1 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 2 tests but ran 1.
Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.03 cusr
0.00 csys = 0.05 CPU)
Result: FAIL
True, pod_coverage_ok() is undefined, because I haven't installed the
library from which it is exported, i.e., Test::Pod::Coverage. But,
without looking at the code in t/pod.t, I would have thought that tests
dependent on Test::Pod::Coverage would be SKIPped. But see this code:
#####
27 my @coverage = qw(
28 File::Path
29 );
30
31 my $test_pod_tests = eval "use Test::Pod"
32 ? 0 : @file;
33
34 my $test_pod_coverage_tests = eval "use Test::Pod::Coverage"
35 ? 0 : @coverage;
36
#####
Assuming I'm using the debugger correctly, when I say 'perl -Iblib/lib
-d t/pod.t' and get to line 31, the string eval at that line and the
other one at line 34 both return 0 -- which means the ternary operator
assigns either scalar(@file) or scalar(@coverage) to $test_pod_tests and
$test_pod_coverage_tests respectively.
#####
main::(t/pod.t:10): if (!$ENV{PERL_AUTHOR_TESTING}) {
DB<1> c 31
main::(t/pod.t:31): my $test_pod_tests = eval "use Test::Pod"
main::(t/pod.t:32): ? 0 : @file;
DB<2> x 'eval "use Test::Pod"'
0 'eval "use Test::Pod"'
DB<3> x 'eval "use Test::Pod::Coverage"'
0 'eval "use Test::Pod::Coverage"'
DB<4> x [@file]
0 ARRAY(0x2f3dfa8)
0 'lib/File/Path.pm'
DB<5> x [@coverage]
0 ARRAY(0x2f3e128)
0 'File::Path'
DB<6> n
main::(t/pod.t:34): my $test_pod_coverage_tests = eval "use
Test::Pod::Coverage"
main::(t/pod.t:35): ? 0 : @coverage;
DB<6> x $test_pod_tests
0 1
DB<7> x 'eval "use Test::Pod::Coverage"'
0 'eval "use Test::Pod::Coverage"'
DB<8> n
1..2
ok 1 - POD test for lib/File/Path.pm
Undefined subroutine &main::pod_coverage_ok called at t/pod.t line 53.
at t/pod.t line 53.
# Looks like you planned 2 tests but ran 1.
# Looks like your test exited with 255 just after 1.
Debugged program terminated.
#####
But this means that subsequently t/pod.t thinks I have
Test::Pod::Coverage installed -- when I don't -- and proceeds not to do
the 'skip' at line 51 and thus to invoke 'pod_coverage_ok()'. This give
the test failure observed above.
Frankly, I think we should junk t/pod.t entirely. Among other things,
when we go to update the version of File-Path in the Perl 5 core, we
have to explicitly exclude t/pod.t from inclusion in the core's test
suite. From Porting/Maintainers.pl:
#####
'File::Path' => {
'DISTRIBUTION' => 'DLAND/File-Path-2.09.tar.gz',
'FILES' => q[cpan/File-Path],
'EXCLUDED' => [
qw( eg/setup-extra-tests
t/pod.t
)
],
'MAP' => {
'' => 'cpan/File-Path/lib/File/',
't/' => 'cpan/File-Path/t/',
},
},
#####
If we are agreed that t/pod.t is more trouble than it is worth, I will
prepare a patch.
Thank you very much.
Jim Keenan