| CC: | Inna Brodkin <innab [...] yahoo-inc.com>, Sundar Rajagopalan <sundarr [...] yahoo-inc.com>, Anirban Kundu <akundu [...] yahoo-inc.com>, Mike Schilli <mschilli [...] yahoo-inc.com> |
| Subject: | Times reported by TAP-Formatter-JUnit-0.06 for individual test cases in a .t file are incorrect |
| Date: | Fri, 22 Jan 2010 15:42:50 -0800 |
| To: | "bug-TAP-Formatter-JUnit [...] rt.cpan.org" <bug-TAP-Formatter-JUnit [...] rt.cpan.org> |
| From: | Marc Abramowitz <marca [...] yahoo-inc.com> |
Hi Graham,
Great module. Thanks for writing it! Like yourself, we are using it so that
we can integrate Perl test results into Hudson builds. I originally tried
using TAP::Harness::JUnit, but that module doesn't support accurate timing
per test case, since it doesn't deal with individual lines of TAP output.
I did, however, run into a small problem with the timing reporting in
TAP-Formatter-JUnit-0.06.
Here's a test that illustrates that the time reported for test case number x
is actually the time for test case number x + 1.
$ cat t/TestDelays.t; prove --timer --formatter TAP::Formatter::JUnit
use Test::More tests => 4;
sleep(0);
ok(1, 'After sleeping for 0 seconds');
sleep(2);
ok(2, 'After sleeping for 2 seconds');
sleep(3);
ok(3, 'After sleeping for 3 seconds');
sleep(4);
ok(4, 'After sleeping for 4 seconds');
<testsuites>
<testsuite failures="0"
errors="0"
time="9.02368998527527"
tests="4"
name="TestDelays_t">
<testcase time="2.01857495307922"
name="1 - After sleeping for 0 seconds"></testcase>
<testcase time="3.00161695480347"
name="2 - After sleeping for 2 seconds"></testcase>
<testcase time="4.00286602973938"
name="3 - After sleeping for 3 seconds"></testcase>
<testcase time="0.000725984573364258"
name="4 - After sleeping for 4 seconds"></testcase>
<system-out><![CDATA[1..4
ok 1 - After sleeping for 0 seconds
ok 2 - After sleeping for 2 seconds
ok 3 - After sleeping for 3 seconds
ok 4 - After sleeping for 4 seconds
]]></system-out>
<system-err></system-err>
</testsuite>
If I change the sleep(0) above to a sleep(1), then the output is:
<testsuites>
<testsuite failures="0"
errors="0"
time="10.0256741046906"
tests="4"
name="TestDelays_t">
<testcase time="3.02052903175354"
name="1 - After sleeping for 1 seconds"></testcase>
<testcase time="3.00264096260071"
name="2 - After sleeping for 2 seconds"></testcase>
<testcase time="4.00192308425903"
name="3 - After sleeping for 3 seconds"></testcase>
<testcase time="0.000674962997436523"
name="4 - After sleeping for 4 seconds"></testcase>
<system-out><![CDATA[1..4
ok 1 - After sleeping for 1 seconds
ok 2 - After sleeping for 2 seconds
ok 3 - After sleeping for 3 seconds
ok 4 - After sleeping for 4 seconds
]]></system-out>
<system-err></system-err>
</testsuite>
This is similar to the previous case in that the times are shifted by 1, but
in this case the first time reported is the time for the 2nd test case + 1
second.
I'm attaching a patch for the Session.pm file in TAP-Formatter-JUnit-0.06
which fixes the problem for me.
With this patch, I get the following output for the second case I mentioned:
<testsuites>
<testsuite failures="0"
errors="0"
time="10.0242660045624"
tests="4"
name="TestDelays_t">
<testcase time="1.00341320037842"
name="1 - After sleeping for 1 seconds"></testcase>
<testcase time="2.00243878364563"
name="2 - After sleeping for 2 seconds"></testcase>
<testcase time="3.0014500617981"
name="3 - After sleeping for 3 seconds"></testcase>
<testcase time="4.00225710868835"
name="4 - After sleeping for 4 seconds"></testcase>
<system-out><![CDATA[1..4
ok 1 - After sleeping for 1 seconds
ok 2 - After sleeping for 2 seconds
ok 3 - After sleeping for 3 seconds
ok 4 - After sleeping for 4 seconds
]]></system-out>
<system-err></system-err>
</testsuite>
</testsuites>
I hope that this patch helps you and thanks again for a great module!
-Marc
Message body not shown because it is not plain text.