Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the IPC-Run-Fused CPAN distribution.

Report information
The Basics
Id: 50979
Status: resolved
Priority: 0/
Queue: IPC-Run-Fused
People
Owner: KENTNL [...] cpan.org
Requestors: ANDK [...] cpan.org
Cc:
AdminCc:
Bug Information
Severity: Normal
Broken in: 0.01000514
Fixed in:
  • 0.01028807
  • 0.02000000
  • 0.03000000

History Show all quoted text
  Fri Oct 30 18:02:53 2009 ANDK [...] cpan.org - Ticket created
Subject: Random test results
In the test script t/01-captures-all.t I find # NOTE: This test is hard to guarantee, its possibly random. Can you explain? If test results are random, what's the point? You may be interested to hear that all of these messages were present in the cpantesters reports: You planned 400 tests but ran 122 You planned 400 tests but ran 123 You planned 400 tests but ran 124 You planned 400 tests but ran 185 You planned 400 tests but ran 54 You planned 400 tests but ran 56 You planned 400 tests but ran 60 You planned 400 tests but ran 89 You planned 400 tests but ran 96 Looks a bit random, right? Thanks && Regards
  Sun Nov 01 21:57:23 2009 KENTNL [...] cpan.org - Correspondence added
On Fri Oct 30 18:02:53 2009, ANDK wrote: Show quoted text
> In the test script t/01-captures-all.t I find > > # NOTE: This test is hard to guarantee, its possibly random. > > Can you explain? If test results are random, what's the point? > > You may be interested to hear that all of these messages were present in > the cpantesters reports: > > You planned 400 tests but ran 122 > You planned 400 tests but ran 123 > You planned 400 tests but ran 124 > You planned 400 tests but ran 185 > You planned 400 tests but ran 54 > You planned 400 tests but ran 56 > You planned 400 tests but ran 60 > You planned 400 tests but ran 89 > You planned 400 tests but ran 96 > > Looks a bit random, right? > > Thanks && Regards
Well, its not the number of runs that is supposed to be random, its the order of the output, its hard to predict. I don't know for certain what order STDERR and STDOUT prints will turn up in the final data stream, so currently am testing for what tends to be consistent on my machine, and getting it tested to see how other platforms behave. The only intentional random is timing jitters in an attempt to introduce concurrency/race condition failures. Inconsistent numbers of test runs *is* a bug however, and due to lack of having physical access to different platforms the initial tests serve partially as an information gathering tool. ( I am aware of these of course, and am trying to find ways to solve them :) ). When things get a bit more sane, the tests might become less extreme.
  Sun Nov 01 21:57:24 2009 The RT System itself - Status changed from 'new' to 'open'
  Sat Nov 07 21:50:30 2009 KENTNL [...] cpan.org - Correspondence added
On Fri Oct 30 18:02:53 2009, ANDK wrote: Show quoted text
> In the test script t/01-captures-all.t I find > > # NOTE: This test is hard to guarantee, its possibly random. > > Can you explain? If test results are random, what's the point? > > You may be interested to hear that all of these messages were present in > the cpantesters reports: > > You planned 400 tests but ran 122 > You planned 400 tests but ran 123 > You planned 400 tests but ran 124 > You planned 400 tests but ran 185 > You planned 400 tests but ran 54 > You planned 400 tests but ran 56 > You planned 400 tests but ran 60 > You planned 400 tests but ran 89 > You planned 400 tests but ran 96 > > Looks a bit random, right? > > Thanks && Regards
I removed the useless calls to POSIX::dup() which I wasn't using anyway, and it uses perl native pipe() command now instead of POSIX::pipe() ( I would think 400 * 2 calls to dup() would result in needless extra filehandle opens which might get ulimited ) these appear to have made the tests pass more ( at least they're working on solaris now, which is a first ) Please test the code yourself and If I don't see any failures from CPANTS in the next 4 weeks I'll mark this particular bug as resolved =)
  Tue Feb 05 02:21:28 2013 KENTNL [...] cpan.org - Taken
  Tue Feb 05 02:22:16 2013 KENTNL [...] cpan.org - Fixed in 0.01028807 added
  Tue Feb 05 02:22:16 2013 KENTNL [...] cpan.org - Fixed in 0.02000000 added
  Tue Feb 05 02:22:16 2013 KENTNL [...] cpan.org - Fixed in 0.03000000 added
  Tue Feb 05 02:22:16 2013 KENTNL [...] cpan.org - Correspondence added
It appears the random test results problem has gone away.

 

If pain persists, please reopen this bug. Thanks.

  Tue Feb 05 02:22:17 2013 KENTNL [...] cpan.org - Status changed from 'open' to 'resolved'