| Subject: | Tested tests that call plan() cause confusing problems |
If any of the tests being tested call plan() it causes lots of (very
confusing) problems.
Defining a dummy plan() function on Test::Tester::Delegate or
Test::Tester::Capture made things work ok for me (no pun intended).
I didn't notice any mention of plan() in Test::Tester
so I didn't know if this was known-behavior or simply hadn't been addressed.
I don't know if the solution is to define that function (and possibly do
something with the values) or to merely document that it may be
necessary. I am not sure how this would affect any subtests that get
run inside of run_tests() and I am not sure if there would be a general
practice that would be useful for most situations. Perhaps there is not
and it should simply be a warning in the documentation that says that
plans may need to be handled specially.
The example where I hit this issue:
I was specifically testing that Test::Spelling was operating on the
files I configured... calling pod_file_spelling_ok() is a simple test
and works just fine. However, all_pod_files_spelling_ok() sets the plan
to the number of files it finds because it assumes it is the only test
being run in the file.
I have attached a test file that demonstrates the issue (and its output).
Any thoughts?
| Subject: | tester-plan-output.txt |
ok 1 - Test 'check single' completed
ok 2 - Test 'check single' no premature diagnostication
ok 3 - Test 'check single' result count
ok 4 - subtest 'single' of 'check single' compare ok
ok 5 - subtest 'single' of 'check single' compare name
ok 6 - checking depth
1..6
ok 1 - check_single
Use of uninitialized value $indent in concatenation (.) or string at /home/rando/perl5/perlbrew/perls/5.14.1-st/lib/5.14.1/Test/Builder.pm line 1754.
not ok 1 - Test 'check all default plan()' completed
# Failed test 'Test 'check all default plan()' completed'
# at t/plan.t line 33.
# Can't use an undefined value as a symbol reference at /home/rando/perl5/perlbrew/perls/5.14.1-st/lib/5.14.1/Test/Builder.pm line 1759.
ok 2 - Test 'check all default plan()' no premature diagnostication
not ok 3 - Test 'check all default plan()' result count
# Failed test 'Test 'check all default plan()' result count'
# at t/plan.t line 33.
# got: 0
# expected: 1
not ok 4 - subtest '' of 'check all default plan()' compare ok
# Failed test 'subtest '' of 'check all default plan()' compare ok'
# at t/plan.t line 33.
# got: undef
# expected: '1'
not ok 5 - subtest '' of 'check all default plan()' compare name
# Failed test 'subtest '' of 'check all default plan()' compare name'
# at t/plan.t line 33.
# got: undef
# expected: 'single'
not ok 6 - checking depth
# Failed test 'checking depth'
# at t/plan.t line 33.
# got: undef
# expected: '2'
# You need to change $Test::Builder::Level
1..6
# Looks like you failed 5 tests of 6.
not ok 2 - check_all_default
# Failed test 'check_all_default'
# at t/plan.t line 36.
ok 1 - Test 'check all dummy plan()' completed
ok 2 - Test 'check all dummy plan()' no premature diagnostication
ok 3 - Test 'check all dummy plan()' result count
ok 4 - subtest 'single' of 'check all dummy plan()' compare ok
ok 5 - subtest 'single' of 'check all dummy plan()' compare name
ok 6 - checking depth
1..6
ok 3 - check_all_dummy
1..3
# Looks like you failed 1 test of 3.
| Subject: | tester-plan-test.pl |
use strict;
use warnings;
use Test::Tester;
use Test::More 0.88;
{
package Test::Tester::TestThingThatCallsPlan;
use Exporter;
our @ISA = qw(Exporter);
our @EXPORT = qw(single_ok all_ok);
my $Test = Test::Builder->new;
sub single_ok { $Test->ok(1, 'single') }
sub all_ok { $Test->plan(tests => 1); single_ok(); }
}
Test::Tester::TestThingThatCallsPlan->import();
subtest check_single => sub {
check_test(
sub {
single_ok();
},
{ ok => 1, name => 'single', depth => 1 },
'check single'
);
};
subtest check_all_default => sub {
check_test(
sub {
all_ok();
},
{ ok => 1, name => 'single', depth => 2 },
'check all default plan()'
);
};
subtest check_all_dummy => sub {
check_test(
sub {
local *Test::Tester::Delegate::plan = sub {};
all_ok();
},
{ ok => 1, name => 'single', depth => 2 },
'check all dummy plan()'
);
};
done_testing;