One of our test frameworks at the BBC has difficulties using TODO
testing due to the declarative nature of the tests being written in
YAML. Attached is a patch with allows us fine-grained control over TODO
tests. It basically lets us do things like this:
my $builder = Test::Builder->new;
foreach my $test_case ($framework->cases) {
if ( my $message = $test_case->todo ) {
$builder->start_todo($message);
$test_case->run;
$builder->end_todo;
}
else {
$test_case->run;
}
}
Going through our test framework and wrapping every possible test in an
optional TODO is rather problematic. This allows us to block out test
cases easily.
We're implementing this today because lack of a feature like this is
causing us much grief.
Cheers,
Ovid
| Subject: | test_builder_todo.patch |
diff -ur Test-Simple-0.80.orig//Changes Test-Simple-0.80/Changes
--- Test-Simple-0.80.orig//Changes Sun Apr 6 16:25:57 2008
+++ Test-Simple-0.80/Changes Tue Jul 29 15:43:25 2008
@@ -1,3 +1,8 @@
+0.08_01 Tue Jul 29 15:42:19 GMT 2008
+ Improved 'todo' support.
+ - TODO tests can now start and end with 'start_todo' and 'end_todo'
+ methods called on the Test::Builder singleton.
+
0.80 Sun Apr 6 17:25:01 CEST 2008
Test fixes
- Completely disable the utf8 test. It was causing perl to panic on some OS's.
Common subdirectories: Test-Simple-0.80.orig//lib and Test-Simple-0.80/lib
Common subdirectories: Test-Simple-0.80.orig//t and Test-Simple-0.80/t
Common subdirectories: Test-Simple-0.80.orig//lib/Test and Test-Simple-0.80/lib/Test
Common subdirectories: Test-Simple-0.80.orig//lib/Test/Builder and Test-Simple-0.80/lib/Test/Builder
diff -ur Test-Simple-0.80.orig//lib/Test/Builder.pm Test-Simple-0.80/lib/Test/Builder.pm
--- Test-Simple-0.80.orig//lib/Test/Builder.pm Sun Apr 6 16:26:10 2008
+++ Test-Simple-0.80/lib/Test/Builder.pm Tue Jul 29 15:47:46 2008
@@ -3,7 +3,7 @@
use 5.006;
use strict;
-our $VERSION = '0.80';
+our $VERSION = '0.80_01';
$VERSION = eval { $VERSION }; # make the alpha version come out as a number
# Make Test::Builder thread-safe for ithreads.
@@ -178,6 +178,8 @@
$self->{No_Ending} = 0;
$self->{TODO} = undef;
+ $self->{TODO_STACK} = [];
+ $self->{START_TODO} = 0;
$self->_dup_stdhandles unless $^C;
@@ -1609,6 +1611,86 @@
: 0;
}
+=item B<start_todo>
+
+ $Test->start_todo($message);
+
+This method allows you declare all subsequent tests as TODO tests, up until
+the C<end_todo> method has been called. Calling it without a message is
+fatal.
+
+The C<TODO:> and C<$TODO> syntax is generally pretty good about figuring out
+whether or not we're in a TODO test. However, often we find that this is not
+possible to determine (such as when we want to use C<$TODO> but
+the tests are being executed in other packages which can't be inferred
+beforehand).
+
+Note that you can use this to nest "todo" tests
+
+ $Test->start_todo('working on this');
+ # lots of code
+ $Test->start_todo('working on that');
+ # more code
+ $Test->end_todo;
+ $Test->end_todo;
+
+This is generally not recommended, but large testing systems often have weird
+internal needs.
+
+We've tried to make this also work with the TODO: syntax, but it's not
+guaranteed and its use is also discouraged:
+
+ TODO: {
+ local $TODO = 'We have work to do!';
+ $Test->start_todo('working on this');
+ # lots of code
+ $Test->start_todo('working on that');
+ # more code
+ $Test->end_todo;
+ $Test->end_todo;
+ }
+
+Pick one style or another of "TODO" to be on the safe side.
+
+=cut
+
+sub start_todo {
+ my ( $self, $message ) = @_;
+ unless ( defined $message ) {
+ $self->diag('start_todo() requires a message!');
+ _my_exit( 255 ) && return;
+ }
+ $self->{START_TODO}++;
+ if ( my $todo = $self->todo ) {
+ push @{ $self->{TODO_STACK} } => $todo;
+ }
+ $self->{TODO} = $message;
+}
+
+=item C<end_todo>
+
+ $Test->end_todo;
+
+Stops running tests as "TODO" tests. This method is fatal if called without a
+preceding C<start_todo> method call.
+
+=cut
+
+sub end_todo {
+ my $self = shift;
+ unless ( $self->{START_TODO}) {
+ $self->diag('end_todo() called without start_todo!');
+ _my_exit( 255 ) && return;
+ }
+ $self->{START_TODO}--;
+ if ( $self->{START_TODO} && @{ $self->{TODO_STACK} } ) {
+ $self->{TODO} = pop @{ $self->{TODO_STACK} };
+ }
+ else {
+ delete $self->{TODO};
+ }
+}
+
=item B<caller>
my $package = $Test->caller;
Common subdirectories: Test-Simple-0.80.orig//lib/Test/Builder/Tester and Test-Simple-0.80/lib/Test/Builder/Tester
Common subdirectories: Test-Simple-0.80.orig//t/lib and Test-Simple-0.80/t/lib
diff -ur Test-Simple-0.80.orig//t/todo.t Test-Simple-0.80/t/todo.t
--- Test-Simple-0.80.orig//t/todo.t Wed Feb 27 09:37:18 2008
+++ Test-Simple-0.80/t/todo.t Tue Jul 29 15:37:54 2008
@@ -9,7 +9,7 @@
use Test::More;
-plan tests => 19;
+plan tests => 30;
$Why = 'Just testing the todo interface.';
@@ -78,11 +78,46 @@
'todo_skip without $how_many warning' );
}
-
+my $builder = Test::More->builder;
+my $exported_to = $builder->exported_to;
TODO: {
- Test::More->builder->exported_to("Wibble");
+ $builder->exported_to("Wibble");
local $TODO = "testing \$TODO with an incorrect exported_to()";
fail("Just testing todo");
}
+
+$builder->exported_to($exported_to);
+
+$builder->start_todo('Expected failures');
+fail('Testing start_todo()');
+ok 0, 'Testing start_todo() with more than one failure';
+$is_todo = $builder->todo;
+$builder->end_todo;
+is $is_todo, 'Expected failures',
+ 'start_todo should have the correct TODO message';
+ok 1, 'end_todo() should not leak TODO behavior';
+
+my @nested_todo;
+my ( $level1, $level2 ) = ( 'failure level 1', 'failure_level 2' );
+TODO: {
+ local $TODO = 'Nesting TODO';
+ fail('fail 1');
+ $builder->start_todo($level1);
+ fail('fail 2');
+ push @nested_todo => $builder->todo;
+ $builder->start_todo($level2);
+ fail('fail 3');
+ push @nested_todo => $builder->todo;
+ $builder->end_todo;
+ fail('fail 4');
+ push @nested_todo => $builder->todo;
+ $builder->end_todo;
+ $is_todo = $builder->todo;
+ fail('fail 4');
+}
+is_deeply \@nested_todo, [ $level1, $level2, $level1 ],
+ 'Nested TODO message should be correct';
+is $is_todo, 'Nesting TODO',
+ '... and original TODO message should be correct';
Common subdirectories: Test-Simple-0.80.orig//t/lib/Dev and Test-Simple-0.80/t/lib/Dev
Common subdirectories: Test-Simple-0.80.orig//t/lib/Test and Test-Simple-0.80/t/lib/Test
Common subdirectories: Test-Simple-0.80.orig//t/lib/Test/Simple and Test-Simple-0.80/t/lib/Test/Simple
Common subdirectories: Test-Simple-0.80.orig//t/lib/Test/Simple/sample_tests and Test-Simple-0.80/t/lib/Test/Simple/sample_tests