Skip Menu |

This queue is for tickets about the TAP-Formatter-JUnit CPAN distribution.

Report information
The Basics
Id: 81552
Status: resolved
Priority: 0/
Queue: TAP-Formatter-JUnit

People
Owner: GTERMARS [...] cpan.org
Requestors: ANDK [...] cpan.org
Cc: gregoa [...] cpan.org
AdminCc:

Bug Information
Severity: Important
Broken in: 0.09
Fixed in: 0.10



Subject: Test failures due to hash randomisation in perl 5.17.6
Since bleadperl v5.17.5-518-g7dc8663 your tests are failing frequently. That commit introduced hash key randomization and it seems at least the tests t/formatter.t t/tap2junit-filter.t t/tap2junit.t t/timer.t are hit by that. Find a sample fail report at: http://www.cpantesters.org/cpan/report/d8da809e-31b8-11e2-9d08-cce7a290f8f5 You can read more about the commit at http://perl5.git.perl.org/perl.git/commit/7dc8663964c66a698d31bbdc8e8abed69bddeec3 You may have to run the test several times until the randomization causes a fail. HTH&&Thanks&&Regards,
From: paul [...] city-fan.org
On Thu Nov 29 17:48:32 2012, ANDK wrote: Show quoted text
> Since bleadperl v5.17.5-518-g7dc8663 your tests are failing > frequently. > That commit introduced hash key randomization and it seems at least > the > tests > > t/formatter.t > t/tap2junit-filter.t > t/tap2junit.t > t/timer.t > > are hit by that. Find a sample fail report at: > > http://www.cpantesters.org/cpan/report/d8da809e-31b8-11e2-9d08- > cce7a290f8f5 > > You can read more about the commit at > http://perl5.git.perl.org/perl.git/commit/7dc8663964c66a698d31bbdc8e8abed69bddeec3
Strictly speaking, this is a test suite issue since the generated XML remains valid, but some attributes are output in a different order than expected by the test suite. However, since I can't think of any simple way to make the tests more forgiving of the order of the attributes in the XML, I've tweaked the code to use Tie::IxHash to ensure that the generated XML has the attributes in the same order as the tests expect them. See attached patch.
Subject: TAP-Formatter-JUnit-0.09-hashorder.patch
This patch addresses https://rt.cpan.org/Public/Bug/Display.html?id=81552 Strictly speaking, this is a test suite issue since the generated XML remains valid, but some attributes are output in a different order than expected by the test suite. However, since I can't think of any simple way to make the tests more forgiving of the order of the attributes in the XML, I've tweaked the code to use Tie::IxHash to ensure that the generated XML has the attributes in the same order as the tests expect them. --- lib/TAP/Formatter/JUnit/Session.pm +++ lib/TAP/Formatter/JUnit/Session.pm @@ -10,6 +10,7 @@ use File::Path qw(mkpath); use IO::File; use TAP::Formatter::JUnit::Result; +use Tie::IxHash; has 'testcases' => ( is => 'rw', @@ -104,10 +105,12 @@ if ($timer_enabled) { unless ($result->is_test) { my $duration = $result->time - $t_start; - my $case = $xml->testcase( { - 'name' => _squeaky_clean('(init)'), + tie my %case_attrs, 'Tie::IxHash'; + %case_attrs = ( 'time' => $duration, - } ); + 'name' => _squeaky_clean('(init)'), + ); + my $case = $xml->testcase( \%case_attrs ); $self->add_testcase($case); $t_last_test = $result->time; } @@ -136,22 +139,23 @@ if ($bogosity) { my $cdata = $self->_cdata($content); my $level = $bogosity->{level}; - $failure = $xml->$level( { + tie my %error_attrs, 'Tie::IxHash'; + %error_attrs = ( type => $bogosity->{type}, message => $bogosity->{message}, - }, $cdata ); + ); + $failure = $xml->$level( \%error_attrs, $cdata ); } # add this test to the XML stream - my $case = $xml->testcase( - { - 'name' => _get_testcase_name($result), + tie my %case_attrs, 'Tie::IxHash'; + %case_attrs = ( ( $timer_enabled ? ('time' => $duration) : () ), - }, - $failure, + 'name' => _get_testcase_name($result), ); + my $case = $xml->testcase( \%case_attrs, $failure, ); $self->add_testcase($case); # update time of last test seen @@ -162,10 +166,12 @@ # track time for teardown, if needed if ($timer_enabled) { my $duration = $self->parser->end_time - $queue->[-1]->time; - my $case = $xml->testcase( { - 'name' => _squeaky_clean('(teardown)'), + tie my %case_attrs, 'Tie::IxHash'; + %case_attrs = ( 'time' => $duration, - } ); + 'name' => _squeaky_clean('(teardown)'), + ); + my $case = $xml->testcase( \%case_attrs ); $self->add_testcase($case); } @@ -225,11 +231,12 @@ } my @tests = @{$self->testcases()}; - my %attrs = ( - 'name' => _get_testsuite_name($self), - 'tests' => $testsrun, + tie my %attrs, 'Tie::IxHash'; + %attrs = ( 'failures' => $failures, 'errors' => $num_errors, + 'tests' => $testsrun, + 'name' => _get_testsuite_name($self), ( $timer_enabled ? ('time' => $time) : () ),
On Thu Nov 29 17:48:32 2012, ANDK wrote: Show quoted text
> Since bleadperl v5.17.5-518-g7dc8663 your tests are failing > frequently. > That commit introduced hash key randomization and it seems at least > the > tests > > t/formatter.t > t/tap2junit-filter.t > t/tap2junit.t > t/timer.t > > are hit by that. Find a sample fail report at: > > http://www.cpantesters.org/cpan/report/d8da809e-31b8-11e2-9d08- > cce7a290f8f5 > > You can read more about the commit at > http://perl5.git.perl.org/perl.git/commit/7dc8663964c66a698d31bbdc8e8abed69bddeec3 > > You may have to run the test several times until the randomization > causes a fail. >
This problem persists. On the basis of this blog post, http://blogs.perl.org/users/confuseacat/2011/09/perl-testing-with-jenkinshudson-avoiding-some-pitfalls.html, I tried to install v0.09 today with 'cpanm'. All prerequisites installed okay. But I got failures in the same 4 files cited by ANDK in Nov 2012. And a very cursory examination suggests that it is a problem first of all with the formatting of the output XML, rather than with its validity. Thank you very much. Jim Keenan
Tests updated in v0.10, and they should no longer expect a given ordering for the XML attributes.