| Subject: | TODO diagnostic display heuristics too easily fooled. |
The existing heuristic used to decide whether a diagonstic is part of a
TODO test is easily fooled and can lead to important failure diagnostics
not being displayed. Here is a simple example.
$| = 1;
print "1..2\n";
print "not ok 1 # TODO\n";
print "# This diagnostic is part of the TODO and should not be
displayed\n";
warn "# This is important information about the fact that the next
test failed.\n";
print "not ok 2\n";
$ prove ~/tmp/fail.t
/Users/schwern/tmp/fail....# This is important information about the
fact that the next test failed.
/Users/schwern/tmp/fail....FAILED test 2
Failed 1/2 tests, 50.00% okay
Failed Test Stat Wstat Total Fail List of Failed
-------------------------------------------------------------------------------
/Users/schwern/tmp/fail.t 2 1 2
Failed 1/1 test scripts. 1/2 subtests failed.
Files=1, Tests=2, 0 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)
Failed 1/1 test programs. 1/2 subtests failed.
$ runtests ~/tmp/fail.t
/Users/schwern/tmp/fail...... Failed 1/2 subtests
Test Summary Report
-------------------
/Users/schwern/tmp/fail.t (Wstat: 0 Tests: 2 Failed: 1)
Failed tests: 2
Files=1, Tests=2, 0 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)