On Wed, Mar 02, 2005 at 04:38:30PM -0500, Guest via RT wrote:
Show quoted text> For developers who use Test::Harness, it would be helpful if we had the ability to tell the system to abort as soon as it saw a discrepancy. It would also be nice to be able to specify verbosity for error messages. I'm doing Bugzilla development currently and the testing suite doesn't tell me the filename having issues making it more difficult for me to locate the problem. The testing suite is Test::Harness. I've found ways around the limitation, however, it would be to everyone's advantage if we could tell the test suite to stop and give us more information about the problem found.
>
> I'm using Test-Harness-2.46 on Perl 5.8.0 under Linux 2.4.20-31.9 (RH9).
First, a clarification.
Test::Harness is not a testing suite. It runs the individual test files,
parses their output and provides a summary of the failure.
The test files themselves are written using a variety of different techniques
and testing libraries, the most popular of them being Test::More.
Regardless, Test::Harness should be reporting which testing file contained
the error like so:
0 ~/tmp$ perl -MTest::Harness -e 'runtests @ARGV' test.t
test....FAILED test 1
Failed 1/1 tests, 0.00% okay
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
test.t 1 1 100.00% 1
Failed 1/1 test scripts, 0.00% okay. 1/1 subtests failed, 0.00% okay.
Additional diagnostic information may be provided by your test programs
assuming they are using a decent testing library such as Test::More.
This has nothing to do with Test::Harness.
0 ~/tmp$ perl -MTest::Harness -e 'runtests @ARGV' new.t
new....NOK 13
# Failed test (new.t at line 41)
# 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)'
# doesn't match '(?-xism:^Mozilla.+compatible.+VMS)'
# Looks like you failed 1 test of 14.
new....dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 13
Failed 1/14 tests, 92.86% okay
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
new.t 1 256 14 1 7.14% 13
Failed 1/1 test scripts, 0.00% okay. 1/14 subtests failed, 92.86% okay.
Given that Test::Harness reports which test file failed we'd need more
information as to what your problem is. Perhaps its a Bugzilla limitation?
