Skip Menu |

This queue is for tickets about the Test-Output CPAN distribution.

Report information
The Basics
Id: 45598
Status: resolved
Priority: 0/
Queue: Test-Output

People
Owner: Nobody in particular
Requestors: jeff.holt [...] method-r.com
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 0.11
Fixed in: (no value)



Subject: Many (all?) of this module's tests are not properly defined
The reason the tests are not properly defined is that they don't test the complete behavior of the print function. For example, the first test in t/combined_is.t (all 3 releases) asserts printing the string "TEST OUT\n" is the string "TEST OUT\n". At first glance, this appears to be a good test definition. But the print function interprets line endings relative to its port. On MSWin32 it should compare it to "TEST OUT\015\012", on non-Mac *NIX ports it should compare it to "TEST OUT\012", and on Mac ports it should compare it to "TEST OUT\015". See "perldoc perlport" for more detail. I doubt that Sorichetti meant to impose this problem and so I recommend that all tests be modified to remove the line ending characters. W.r.t. the aforementioned test, here is what I think it should look like: check_test( sub { combined_is(sub { print "TEST OUT"; }, "TEST OUT", 'Testing STDOUT' ) },{ ok => 1, name => 'Testing STDOUT', diag => '', },'STDOUT matches success' ); But even if Sorichetti meant to handle line endings, we might be able to address it in a port independent way by setting $/ to a fixed value before the tests begin. And, of course, if Sorichetti meant to handle line endings natively (which I doubt), then we can create a test data folder to hold files for each supported port (i.e., $^O). Upon agreement, I'll provide the patch.
There is one more reason tie'ing is invalid. On some (all?) platforms, the following sequence of prints by a single process produces unintuitive results. perl -e "print qq(1); print STDERR qq(2); print qq(3)" It prints "213" not "123" on the following platforms: 5.10.0 on MSWin32 5.10.0 on Linux x86 5.8.8 on Linux x86 5.10.0 on SPARC Solaris 10 5.8.4 on SPARC Solaris 10 Tie'd printing disrupts the natural way in which output is rendered. This, just like disrupting line ending behavior, makes the tests invalid because the tested code doesn't behave the same way it does when it isn't tested. That's bad. Real bad.
Hi Jeff, This is a problem we have to figure out, but it's not a problem with the implementation of the module I don't think. When you print to any set of buffered filehandles, you can't know necessarily when they will each flush, so no order is guaranteed. This does present some challenges for combined output, and without looking at the code at the moment, I'm thinking that we'll have to autoflush both STDOUT and STDERR.
From: jeff.holt [...] method-r.com
"When you print to any set of buffered filehandles, you can't know necessarily when they will each flush, so no order is guaranteed." Yes, that's correct. And that's why tie'ing is not proper: it orders the output. Ordering makes tested code behave in a way that is different than when it is not tested. Maybe I made a mistaken in the title of this issue. I thought the best thing to do was to bug the effect (bad tests), not the cause (tie'ing). But I suppose nobody (even me) really cares whether this module's tests are valid. All they care about is whether this module can properly test their application.
We no longer use ties in Test::Output, so this shouldn't be a problem. If Capture::Tiny ends up having the same problem, we can open a new ticket.