| Subject: | t/WebBenchmark.xt may fail |
Sometimes t/WebBenchmark.xt fails:
# t/testcases/Web/WebBenchmark.pm:144 - testcases::Web::WebBenchmark(test_all)
# Boolean assertion failed
# Expected to render into '5|(test:1:1:0)(p:/bits/complex-template:30:1:0)(p:/bits/test-recurring:20:1:0)(p:/bits/test-non-cacheable:10:0:0)(p:/bits/system-test:10:1:1)|5', got '5|(test:1:1:0)(p:/bits/test-recurring:20:1:0)(p:/bits/complex-template:30:1:0)(p:/bits/test-non-cacheable:10:0:0)(p:/bits/system-test:10:1:1)|5'
# at t/testcases/Web/WebBenchmark.pm line 144.
# testcases::Web::WebBenchmark::test_all(testcases::Web::WebBenchmark=HASH(0x41cd2750)) called at /usr/perl5.24.1p/lib/site_perl/5.24.1/Test/Unit/Lite.pm line 602
# eval {...} called at /usr/perl5.24.1p/lib/site_perl/5.24.1/Test/Unit/Lite.pm line 601
# Test::Unit::TestSuite::run(Test::Unit::TestSuite=HASH(0x41cc1288), Test::Unit::Result=HASH(0x41cb3af8), Test::Unit::HarnessUnit=HASH(0x41cb35e8)) called at /usr/perl5.24.1p/lib/site_perl/5.24.1/Test/Unit/Lite.pm line 750
# Test::Unit::TestRunner::start(Test::Unit::HarnessUnit=HASH(0x41cb35e8), "testcases::Web::WebBenchmark") called at t/WebBenchmark.xt line 10
t/WebBenchmark.xt .....
Failed 1/1 subtests
It seems that the expected order is not the same as the got one. My first thought was "hash randomization", but the problem appears also for perls where hash randomization is not implemented (see http://www.cpantesters.org/cpan/report/02910c14-d6d6-11e8-82b0-80287347484a for an example)