Skip Menu |

This queue is for tickets about the Test-Harness CPAN distribution.

Report information
The Basics
Id: 72798
Status: open
Priority: 0/
Queue: Test-Harness

People
Owner: Nobody in particular
Requestors: mschwern [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 3.23
Fixed in: (no value)



Subject: Subtests are ignored
It appears that subtest TAP is ignored. $ cat > ~/test.plx print <<END; 1..2 ok 1 1..3 ok 1 ok 2 not ok 3 ok 2 END $ prove ~/tmp/test.plx /Users/schwern/tmp/test.plx .. ok All tests successful. Files=1, Tests=2, 0 wallclock secs ( 0.02 usr + 0.01 sys = 0.03 CPU) Result: PASS That should be a fail. Outputting a test after the subtest is only a convention for compatibility.
On Tue Nov 29 02:25:31 2011, MSCHWERN wrote: Show quoted text
> It appears that subtest TAP is ignored. > > $ cat > ~/test.plx > print <<END; > 1..2 > ok 1 > 1..3 > ok 1 > ok 2 > not ok 3 > ok 2 > END > > $ prove ~/tmp/test.plx > /Users/schwern/tmp/test.plx .. ok > All tests successful. > Files=1, Tests=2, 0 wallclock secs ( 0.02 usr + 0.01 sys = 0.03 CPU) > Result: PASS > > That should be a fail. Outputting a test after the subtest is only a > convention for compatibility.
Is it? I think the TAP standardization process should figure that out, but sadly that seems to have been stalled. I have no idea what to do with this now. Leon
On Wed Nov 13 09:26:23 2013, LEONT wrote: Show quoted text
> On Tue Nov 29 02:25:31 2011, MSCHWERN wrote:
> > It appears that subtest TAP is ignored. > > > > $ cat > ~/test.plx > > print <<END; > > 1..2 > > ok 1 > > 1..3 > > ok 1 > > ok 2 > > not ok 3 > > ok 2 > > END > > > > $ prove ~/tmp/test.plx > > /Users/schwern/tmp/test.plx .. ok > > All tests successful. > > Files=1, Tests=2, 0 wallclock secs ( 0.02 usr + 0.01 sys = 0.03 > > CPU) > > Result: PASS > > > > That should be a fail. Outputting a test after the subtest is only a > > convention for compatibility.
> > Is it? I think the TAP standardization process should figure that out, > but sadly that seems to have been stalled. > > I have no idea what to do with this now. > > Leon
Agreed, that should be a fail. What's going on is that both AndyA and myself found that parsing a subtest is harder than it looks. I don't recall everything involved, but I seem to recall looking at TAP::Parser::Grammar::tokenize and concluding it would need to be able to return a new token type, TAP::Parser::Result::Parser, based on indentation. That would allow syntax highlighting and make it easier to generate proper statistics. My first pass at doing this failed miserably, but I don't remember why. Both of us eventually ran out of tuits. Fortunately, the concern, at this point, is theoretical. It can become practical if TAP is generated from aggregating different TAP streams instead of relying on subtest() to generate the actual TAP. Cheers, Ovid