Bug #108977 for Hypothesis-API: t/search.t sometimes hangs

Sun Nov 15 14:56:35 2015 SREZIC [...] cpan.org - Ticket created

Subject:

t/search.t sometimes hangs

Normally the whole test suite completes in 6 to 18 seconds, but sometimes it simply hangs in t/search.t. I observed this on linux and freebsd systems. Some fail reports in http://matrix.cpantesters.org/?dist=Hypothesis-API%200.12 are because of these hanging tests.

Sun Nov 15 15:02:05 2015 brandon.barker [...] cornell.edu - Correspondence added

Subject:	Re: [rt.cpan.org #108977] t/search.t sometimes hangs
Date:	Sun, 15 Nov 2015 15:01:35 -0500
To:	<bug-Hypothesis-API [...] rt.cpan.org>
From:	Brandon Barker <brandon.barker [...] cornell.edu>

I should probably transition these tests to use some kind of mock interface instead of relying on the hypothesis server. On Nov 15, 2015 2:56 PM, "Slaven_Rezic via RT" < bug-Hypothesis-API@rt.cpan.org> wrote: Show quoted text

> Sun Nov 15 14:56:35 2015: Request 108977 was acted upon. > Transaction: Ticket created by SREZIC > Queue: Hypothesis-API > Subject: t/search.t sometimes hangs > Broken in: 0.12 > Severity: (no value) > Owner: Nobody > Requestors: SREZIC@cpan.org > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=108977 > > > > Normally the whole test suite completes in 6 to 18 seconds, but sometimes > it simply hangs in t/search.t. I observed this on linux and freebsd > systems. Some fail reports in > http://matrix.cpantesters.org/?dist=Hypothesis-API%200.12 are because of > these hanging tests. > >

Sun Nov 15 15:02:05 2015 The RT System itself - Status changed from 'new' to 'open'

Sat Nov 28 21:38:39 2015 brandon.barker [...] cornell.edu - Correspondence added

Subject:	Re: [rt.cpan.org #108977] t/search.t sometimes hangs
Date:	Sat, 28 Nov 2015 21:38:20 -0500
To:	<bug-Hypothesis-API [...] rt.cpan.org>
From:	Brandon Barker <brandon.barker [...] cornell.edu>

After thinking about this a bit more, I believe I will simply use a short timeout in testing code and try to fail with a more informative error. I believe it is important to test the ability to interact with the hypothes.is server, as there is no better (or even good) way for me to stay informed of REST API changes that may break the software - making the CPAN testers even more valuable in this case (thanks!). Is it possible to ask CPAN testers to not run tests if a service is determined to be down, and wait until the next run? If so, I will try to follow up with Hypothes.is folks to see why this might only be occurring during calls happening in search.t., in order to figure out what is the best way to test for service availability. On Sun, Nov 15, 2015 at 3:02 PM, Brandon Barker via RT < bug-Hypothesis-API@rt.cpan.org> wrote: Show quoted text

> Queue: Hypothesis-API > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=108977 > > > I should probably transition these tests to use some kind of mock interface > instead of relying on the hypothesis server. > On Nov 15, 2015 2:56 PM, "Slaven_Rezic via RT" < > bug-Hypothesis-API@rt.cpan.org> wrote: >

> > Sun Nov 15 14:56:35 2015: Request 108977 was acted upon. > > Transaction: Ticket created by SREZIC > > Queue: Hypothesis-API > > Subject: t/search.t sometimes hangs > > Broken in: 0.12 > > Severity: (no value) > > Owner: Nobody > > Requestors: SREZIC@cpan.org > > Status: new > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=108977 > > > > > > > Normally the whole test suite completes in 6 to 18 seconds, but sometimes > > it simply hangs in t/search.t. I observed this on linux and freebsd > > systems. Some fail reports in > > http://matrix.cpantesters.org/?dist=Hypothesis-API%200.12 are because of > > these hanging tests. > > > >

> >

-- Brandon E. Barker http://www.cac.cornell.edu/barker/

Tue Dec 15 14:41:41 2015 BBARKER [...] cpan.org - Correspondence added

I'm not sure if I should mark this as fixed, but in any event, everything has been green for a while in 0.13. But, it is still expected we may see some red if testers run when the H server is down.

Tue Dec 15 14:41:42 2015 BBARKER [...] cpan.org - Status changed from 'open' to 'resolved'

Tue Dec 15 14:41:52 2015 BBARKER [...] cpan.org - Fixed in 0.13 added

Tue Dec 15 16:12:56 2015 SREZIC [...] cpan.org - Correspondence added

On 2015-11-28 21:38:39, brandon.barker@cornell.edu wrote: [...] Show quoted text

> Is it possible to ask CPAN testers to not run tests if a service is > determined to be down, and wait until the next run?

No, I am not aware of such a mechanism. If all test scripts are skipped (e.g. using plan skip_all => ...), then the overall result is NOTESTS which would yield a test report with state UNKNOWN. This is visible distinguishable on the CPAN Testers Matrix page. Usually a version of a distribution is tested only once per perl version, so on most smoker setups there would be no other run for this distribution version. Another possibility to check for unusual conditions during the test runs would be by using CPAN::Testers::ParseReport. This module can download test reports for a distribution and parse it for some predefined parameters, and it's also possible to search for custom strings. Or, as the reports are locally cached after download, it's also possible to do a simple "grep" on the files. Regards, Slaven