Bug #121643 for TestRail-API: t/testrail-results.t fails (with MCE 1.829?)

Fri May 12 17:58:18 2017 SREZIC [...] cpan.org - Ticket created

Subject:

t/testrail-results.t fails (with MCE 1.829?)

Some of my smokers report test failures like this: ... Not an ARRAY reference at /home/cpansand/.cpan/build/2017051221/TestRail-API-0.039-2/blib/lib/TestRail/Utils/Find.pm line 349, <__ANONIO__> line 2. # Failed test 'Gets correct # of runs with test inside it when filtering by plan name' # at t/testrail-results.t line 79. # '############################# # t/fake.test was present in 252 runs. # Total time spent running this test: 756 seconds # Median time spent running this test: 1.5 seconds # Mean time spent running this test: 1.5 seconds # Standard deviations of runtime in test: 0.500496771111107 # Maximum time spent running this test: 2 seconds # Minimum time spent running this test: 1 seconds # Num times this test has been executed: 504 # Retest: 252 # Failed: 252 # #############################' # doesn't match '(?^:fake\.test was present in 253 runs)' # Looks like you failed 1 test of 33. t/testrail-results.t ............ Dubious, test returned 1 (wstat 256, 0x100) Failed 1/33 subtests ... This seems to happen only if MCE 1.829 is installed, not for older versions. Statistical analysis (negative theta means "bad"): **************************************************************** Regression 'mod:MCE::Loop' **************************************************************** Name Theta StdErr T-stat [0='const'] 1.0000 0.0000 41739314998653728.00 [1='eq_1.810'] 0.0000 0.0000 2.93 [2='eq_1.812'] 0.0000 0.0000 2.93 [3='eq_1.814'] 0.0000 0.0000 4.18 [4='eq_1.819'] 0.0000 0.0000 4.45 [5='eq_1.821'] 0.0000 0.0000 4.63 [6='eq_1.827'] 0.0000 0.0000 3.66 [7='eq_1.829'] -1.0000 0.0000 -39668290604344104.00 R^2= 1.000, N= 90, K= 8 ****************************************************************

Tue May 16 10:50:55 2017 TEODESIAN [...] cpan.org - Correspondence added

cloned to github as https://github.com/teodesian/TestRail-Perl/issues/135 Seems there is some inconsistency with MCE, it might even be losing data here. On Fri May 12 17:58:18 2017, SREZIC wrote: Show quoted text

> Some of my smokers report test failures like this: > > ... > Not an ARRAY reference at > /home/cpansand/.cpan/build/2017051221/TestRail-API-0.039- > 2/blib/lib/TestRail/Utils/Find.pm line 349, <__ANONIO__> line 2. > > # Failed test 'Gets correct # of runs with test inside it when > filtering by plan name' > # at t/testrail-results.t line 79. > # '############################# > # t/fake.test was present in 252 runs. > # Total time spent running this test: 756 seconds > # Median time spent running this test: 1.5 seconds > # Mean time spent running this test: 1.5 seconds > # Standard deviations of runtime in test: 0.500496771111107 > # Maximum time spent running this test: 2 seconds > # Minimum time spent running this test: 1 seconds > # Num times this test has been executed: 504 > # Retest: 252 > # Failed: 252 > # #############################' > # doesn't match '(?^:fake\.test was present in 253 runs)' > # Looks like you failed 1 test of 33. > t/testrail-results.t ............ > Dubious, test returned 1 (wstat 256, 0x100) > Failed 1/33 subtests > ... > > This seems to happen only if MCE 1.829 is installed, not for older > versions. Statistical analysis (negative theta means "bad"): > > **************************************************************** > Regression 'mod:MCE::Loop' > **************************************************************** > Name Theta StdErr T-stat > [0='const'] 1.0000 0.0000 41739314998653728.00 > [1='eq_1.810'] 0.0000 0.0000 2.93 > [2='eq_1.812'] 0.0000 0.0000 2.93 > [3='eq_1.814'] 0.0000 0.0000 4.18 > [4='eq_1.819'] 0.0000 0.0000 4.45 > [5='eq_1.821'] 0.0000 0.0000 4.63 > [6='eq_1.827'] 0.0000 0.0000 3.66 > [7='eq_1.829'] -1.0000 0.0000 -39668290604344104.00 > > R^2= 1.000, N= 90, K= 8 > ****************************************************************

Tue May 16 10:50:56 2017 The RT System itself - Status changed from 'new' to 'open'

Tue May 16 23:26:42 2017 TEODESIAN [...] cpan.org - Correspondence added

I think I've fixed it -- Will resolve if CPANTesters looks ok for v0.040 On Tue May 16 10:50:55 2017, TEODESIAN wrote: Show quoted text

> cloned to github as > https://github.com/teodesian/TestRail-Perl/issues/135 > > Seems there is some inconsistency with MCE, it might even be losing > data here. > > On Fri May 12 17:58:18 2017, SREZIC wrote:

> > Some of my smokers report test failures like this: > > > > ... > > Not an ARRAY reference at > > /home/cpansand/.cpan/build/2017051221/TestRail-API-0.039- > > 2/blib/lib/TestRail/Utils/Find.pm line 349, <__ANONIO__> line 2. > > > > # Failed test 'Gets correct # of runs with test inside it when > > filtering by plan name' > > # at t/testrail-results.t line 79. > > # '############################# > > # t/fake.test was present in 252 runs. > > # Total time spent running this test: 756 seconds > > # Median time spent running this test: 1.5 seconds > > # Mean time spent running this test: 1.5 seconds > > # Standard deviations of runtime in test: 0.500496771111107 > > # Maximum time spent running this test: 2 seconds > > # Minimum time spent running this test: 1 seconds > > # Num times this test has been executed: 504 > > # Retest: 252 > > # Failed: 252 > > # #############################' > > # doesn't match '(?^:fake\.test was present in 253 runs)' > > # Looks like you failed 1 test of 33. > > t/testrail-results.t ............ > > Dubious, test returned 1 (wstat 256, 0x100) > > Failed 1/33 subtests > > ... > > > > This seems to happen only if MCE 1.829 is installed, not for older > > versions. Statistical analysis (negative theta means "bad"): > > > > **************************************************************** > > Regression 'mod:MCE::Loop' > > **************************************************************** > > Name Theta StdErr T-stat > > [0='const'] 1.0000 0.0000 41739314998653728.00 > > [1='eq_1.810'] 0.0000 0.0000 2.93 > > [2='eq_1.812'] 0.0000 0.0000 2.93 > > [3='eq_1.814'] 0.0000 0.0000 4.18 > > [4='eq_1.819'] 0.0000 0.0000 4.45 > > [5='eq_1.821'] 0.0000 0.0000 4.63 > > [6='eq_1.827'] 0.0000 0.0000 3.66 > > [7='eq_1.829'] -1.0000 0.0000 -39668290604344104.00 > > > > R^2= 1.000, N= 90, K= 8 > > ****************************************************************

Tue May 16 23:26:43 2017 TEODESIAN [...] cpan.org - Status changed from 'open' to 'patched'

Mon Jun 05 01:10:10 2017 MARIOROY [...] cpan.org - Correspondence added

Recent versions of MCE added support for a hash as input data. That is causing problems when processing an array ref containing nested hashes: e.g. array of hashes. The solution is to pass the array ref directly to mce_loop so that MCE treats the input as an array ref and not think the first hash structure inside the array to be the input data. Basically, this phenomenon occurs when there is one hash element inside the array causing MCE to think to process a hash for input. Do this instead: mce_loop { ... } $runs Versus: mce_loop { ... } @$runs Kind regards, Mario.

Mon Jun 05 01:22:39 2017 MARIOROY [...] cpan.org - Correspondence added

Please see a thread on Perlmonks where a monk experienced the same issue with processing an array of hashes. http://www.perlmonks.org/?node_id=1191783 When the input data contains 1 element, determining the input type is problematic for nested data structure. In that case for your module, pass $runs instead of @$runs to mce_loop. Kind regards, Mario

Mon Jun 05 08:53:07 2017 TEODESIAN [...] cpan.org - Correspondence added

I've reopened the issue on GitHub and will make a patch along the lines of what you suggest. Thanks for the advice! On Mon Jun 05 01:22:39 2017, MARIOROY wrote: Show quoted text

> Please see a thread on Perlmonks where a monk experienced the same > issue with processing an array of hashes. > > http://www.perlmonks.org/?node_id=1191783 > > When the input data contains 1 element, determining the input type is > problematic for nested data structure. In that case for your module, > pass $runs instead of @$runs to mce_loop. > > Kind regards, > > Mario