Bug #122189 for Test-Smoke: perl5.test-smoke.org: "Latest version" excessively filters out reports

Fri Jun 23 08:42:38 2017 jkeenan [...] cpan.org - Ticket created

Subject:

perl5.test-smoke.org: "Latest version" excessively filters out reports

As reported on irc.perl.org #smoke at 07:57:19 AM EDT on Fri Jun 23 2017: At http://perl5.test-smoke.org/search, the "Latest version" filter seems to be doing excessive filtering. When I search with "All versions", I get a full (25-reports) page of reports each of which has v5.27.1- in the "Git-id" column. And when I go to the 2nd page, there are v5.27.1 reports on that page as well. However, when I switch to "Latest version only" for "Perl version", I get only 8 reports for v5.27.1. In particular, I fail to get a report I generated yesterday for FreeBSD-10.3-RELEASE at v5.27.1-57-g04849b3 -- the one where Host is "localhost". All of the above was observed with no other filters selected. However, when I select FreeBSD for the operating system filter, I get similar results. That is, where I would expect to see a FreeBSD-10.3-RELEASE report for v5.27.1, I do not. Any ideas?

Fri Jun 23 08:48:21 2017 jkeenan [...] cpan.org - Correspondence added

On Fri Jun 23 08:42:38 2017, JKEENAN wrote: Show quoted text

> As reported on irc.perl.org #smoke at 07:57:19 AM EDT on Fri Jun 23 > 2017: > > At http://perl5.test-smoke.org/search, the "Latest version" filter > seems to be doing excessive filtering. > > When I search with "All versions", I get a full (25-reports) page of > reports each of which has v5.27.1- in the "Git-id" column. And when I > go to the 2nd page, there are v5.27.1 reports on that page as well. > > However, when I switch to "Latest version only" for "Perl version", I > get only 8 reports for v5.27.1. In particular, I fail to get a report > I generated yesterday for FreeBSD-10.3-RELEASE at v5.27.1-57-g04849b3 > -- the one where Host is "localhost". > > All of the above was observed with no other filters selected. > However, when I select FreeBSD for the operating system filter, I get > similar results. That is, where I would expect to see a FreeBSD-10.3- > RELEASE report for v5.27.1, I do not. > > Any ideas?

Adding some screenshots which, in the case where I filter for FreeBSD operating system, show that "Latest version only" is not returning the same number of v5.27.1 reports as "All versions".

Subject:

freebsd-all-versions.png

Download freebsd-all-versions.png
image/png 60k

Subject:

freebsd-latest-version-only.png

Download freebsd-latest-version-only.png
image/png 41.3k

Mon Jun 26 14:32:31 2017 abe.timmerman [...] xs4all.nl - Correspondence added

Subject:	Re: [rt.cpan.org #122189] perl5.test-smoke.org: "Latest version" excessively filters out reports
Date:	Mon, 26 Jun 2017 20:31:16 +0200
To:	bug-Test-Smoke [...] rt.cpan.org
From:	Abe Timmerman <abe.timmerman [...] xs4all.nl>

Subject:	Re: [rt.cpan.org #122189] perl5.test-smoke.org: "Latest version" excessively filters out reports
To:	bug-Test-Smoke [...] rt.cpan.org
From:	Abe Timmerman <abe.timmerman [...] xs4all.nl>

On 23/06/2017 14:42, James E Keenan via RT wrote: Hi Jim, [snip] Show quoted text

> As reported on irc.perl.org #smoke at 07:57:19 AM EDT on Fri Jun 23 2017: > > At http://perl5.test-smoke.org/search, the "Latest version" filter seems to be doing excessive filtering.

Well... it depends on one's definition of "latest" I guess ;) In the case of CoreSmokeDB, "latest version" means the version that sorts as "latest/newest" over all smoke-reports registered in the database. In your case 'v5.27.1' doesn't seem to be the "latest" registered version (if you look at the dropdown, you'll see it is v5.27.2 or even higher). If you choose specifics like OS, "latest" will be determined over that resultset... Show quoted text

> When I search with "All versions", I get a full (25-reports) page of reports each of which has v5.27.1- in the "Git-id" column. And when I go to the 2nd page, there are v5.27.1 reports on that page as well. > > However, when I switch to "Latest version only" for "Perl version", I get only 8 reports for v5.27.1. In particular, I fail to get a report I generated yesterday for FreeBSD-10.3-RELEASE at v5.27.1-57-g04849b3 -- the one where Host is "localhost". > > All of the above was observed with no other filters selected. However, when I select FreeBSD for the operating system filter, I get similar results. That is, where I would expect to see a FreeBSD-10.3-RELEASE report for v5.27.1, I do not. > > Any ideas?

It all depends on what question you are trying to answer I guess... Thanks for your interest in Test::Smoke! Abe. -- \|||/ | "Are you pondering what I'm pondering, Pinky?" (o o) | Sure, Brain, but how are we going to find chaps our +-ooO-( )-Ooo-+ size? (10 Nov 93 - Spell-Bound) |

Download signature.asc
application/pgp-signature 181b

Message body not shown because it is not plain text.

Mon Jun 26 14:32:31 2017 The RT System itself - Status changed from 'new' to 'open'

Mon Jun 26 14:55:40 2017 jkeenan [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #122189] perl5.test-smoke.org: "Latest version" excessively filters out reports
Date:	Mon, 26 Jun 2017 14:55:24 -0400
To:	bug-Test-Smoke [...] rt.cpan.org
From:	James E Keenan <jkeenan [...] pobox.com>

On 06/26/2017 02:32 PM, Abe Timmerman via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=122189 > > > On 23/06/2017 14:42, James E Keenan via RT wrote: > > Hi Jim, > > [snip] >

>> As reported on irc.perl.org #smoke at 07:57:19 AM EDT on Fri Jun 23 2017: >> >> At http://perl5.test-smoke.org/search, the "Latest version" filter seems to be doing excessive filtering.

> > Well... it depends on one's definition of "latest" I guess ;) > > In the case of CoreSmokeDB, "latest version" means the version that sorts as "latest/newest" > over all smoke-reports registered in the database. > > In your case 'v5.27.1' doesn't seem to be the "latest" registered version (if you look at the > dropdown, you'll see it is v5.27.2 or even higher). >

Why would v5.27.2 appear in the drop-down when we haven't released that yet? When I select no filter except "Perl version", which I set to v5.27.2, all results come back with "Git-id" starting "v5.27.1-". Show quoted text

> If you choose specifics like OS, "latest" will be determined over that resultset... >

>> When I search with "All versions", I get a full (25-reports) page of reports each of which has v5.27.1- in the "Git-id" column. And when I go to the 2nd page, there are v5.27.1 reports on that page as well. >> >> However, when I switch to "Latest version only" for "Perl version", I get only 8 reports for v5.27.1. In particular, I fail to get a report I generated yesterday for FreeBSD-10.3-RELEASE at v5.27.1-57-g04849b3 -- the one where Host is "localhost". >> >> All of the above was observed with no other filters selected. However, when I select FreeBSD for the operating system filter, I get similar results. That is, where I would expect to see a FreeBSD-10.3-RELEASE report for v5.27.1, I do not. >>

I think this problem still persists. For example, when I select FreeBSD as Operating System and select "All versions", I get 433 results paginated into 18 pages. On the first such page, I get 12 reports whose Git-id starts with "v5.27.1-". (None of these results are "v5.27.2-".) When, I then apply the "Latest Version only" filter in addition to the OS filter, I get only 11 results, each of whose "Git-id" starts with v5.27.1. To further complicate matters, when I select FreeBSD as the OS and select "v5.27.1" as the Perl Version, I get back 19 results: 1 with v5.27.1- in its Git-id and 18 with v5.27.0-. Thanks for looking into this. jimk Show quoted text

>> Any ideas?

> > It all depends on what question you are trying to answer I guess... > > > Thanks for your interest in Test::Smoke! > > > Abe. >

Mon Jun 26 15:44:50 2017 abe.timmerman [...] xs4all.nl - Correspondence added

Subject:	Re: [rt.cpan.org #122189] perl5.test-smoke.org: "Latest version" excessively filters out reports
Date:	Mon, 26 Jun 2017 21:44:31 +0200
To:	bug-Test-Smoke [...] rt.cpan.org
From:	Abe Timmerman <abe.timmerman [...] xs4all.nl>

Subject:	Re: [rt.cpan.org #122189] perl5.test-smoke.org: "Latest version" excessively filters out reports
To:	bug-Test-Smoke [...] rt.cpan.org
From:	Abe Timmerman <abe.timmerman [...] xs4all.nl>

On 26/06/2017 20:55, James E Keenan via RT wrote: Hi Jim, Thanks for your quick response. Show quoted text

> Why would v5.27.2 appear in the drop-down when we haven't released that yet?

Well... (from patchlevel.h) 16 #define PERL_REVISION 5 /* age */ 17 #define PERL_VERSION 27 /* epoch */ 18 #define PERL_SUBVERSION 2 /* generation */ Show quoted text

> When I select no filter except "Perl version", which I set to v5.27.2, > all results come back with "Git-id" starting "v5.27.1-".

Ah, the difference between the version as stated in `patchlevel.h` and the last git-TAG applied for a "release". This will always be a problem... At some stage $pumpking decides to update the version in `patchlevel.h` right before (or right after) a new release. The new release warrants a new tag in git, and that's what you see in 'Git-id'. One hopes there is a relationship between the two, but apart from best-practices there really isn't :( Please look at `Porting/make_dot_patch.pl` to see how `.patch` is generated from a git-repository (that process is also used to generate .patch for the rsync option). There is a difference between the version derived from the last git tag, and the version compiled in by way of patchlevel.h. I'm afraid the latter wins... I hope this helps with this part of the problem. Thanks for your interest in Test::Smoke! good luck, Abe. -- \|||/ | "Are you pondering what I'm pondering?" (o o) | Oooh, I think so Brain, but I think I'd rather eat the +-ooO-( )-Ooo-+ Macarana. (28 Sep 96 - Plan Brain from Outer Space) |

Download signature.asc
application/pgp-signature 181b

Message body not shown because it is not plain text.

Mon Jun 26 16:07:47 2017 jkeenan [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #122189] perl5.test-smoke.org: "Latest version" excessively filters out reports
Date:	Mon, 26 Jun 2017 16:07:30 -0400
To:	bug-Test-Smoke [...] rt.cpan.org
From:	James E Keenan <jkeenan [...] pobox.com>

On 06/26/2017 03:44 PM, Abe Timmerman via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=122189 > > > On 26/06/2017 20:55, James E Keenan via RT wrote: > > Hi Jim, > > Thanks for your quick response. >

>> Why would v5.27.2 appear in the drop-down when we haven't released that yet?

> > Well... (from patchlevel.h) > > 16 #define PERL_REVISION 5 /* age */ > 17 #define PERL_VERSION 27 /* epoch */ > 18 #define PERL_SUBVERSION 2 /* generation */ >

>> When I select no filter except "Perl version", which I set to v5.27.2, >> all results come back with "Git-id" starting "v5.27.1-".

> > Ah, the difference between the version as stated in `patchlevel.h` and the last git-TAG applied > for a "release". > > This will always be a problem... At some stage $pumpking decides to update the version in > `patchlevel.h` right before (or right after) a new release. The new release warrants a new tag > in git, and that's what you see in 'Git-id'. One hopes there is a relationship between the two, > but apart from best-practices there really isn't :( > > Please look at `Porting/make_dot_patch.pl` to see how `.patch` is generated from a > git-repository (that process is also used to generate .patch for the rsync option). > > There is a difference between the version derived from the last git tag, and the version > compiled in by way of patchlevel.h. I'm afraid the latter wins... >

Perhaps http://perl5.test-smoke.org/search could have a footnote that explains that the version numbers under "Perl version" -- including "Latest version only" -- do not directly correspond to the part of "Git-id" up to the first hyphen. This discordance is the source of my original confusion. Thank you very much. jimk

Mon Sep 25 14:15:59 2017 abeltje [...] cpan.org - Correspondence added

I've created an issue on GitHub for this: https://github.com/abeltje/CoreSmokeDB/issues/4 I'll close this ticket and we can all watch the github-issue Thanks for your interset in CoreSmokeDB! Good luck, -- Abe.

Mon Sep 25 14:16:00 2017 abeltje [...] cpan.org - Status changed from 'open' to 'resolved'