Skip Menu |

This queue is for tickets about the Module-CPANTS-Analyse CPAN distribution.

Report information
The Basics
Id: 87023
Status: rejected
Priority: 0/
Queue: Module-CPANTS-Analyse

People
Owner: Nobody in particular
Requestors: ether [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: (no value)
Fixed in: (no value)



Subject: Need more flags on metrics, to make it easier to skip them
On 2013-07-18 10:47:27, ETHER wrote: Show quoted text
> More info, and patch, in https://github.com/daxim/Module-CPANTS- > Analyse/pull/13
This would let you keep the metrics you just removed in 0.90_01: buildtool_not_executable no_dot_underscore_files no_generated_files ...because you can just tag them with something like 'no_build'.
No. Removing them from MCA is better because other users (like T::K::Extra) may suffer the same issue. On Fri Aug 02 08:08:15 2013, ETHER wrote: Show quoted text
> On 2013-07-18 10:47:27, ETHER wrote:
> > More info, and patch, in https://github.com/daxim/Module-CPANTS- > > Analyse/pull/13
> > > This would let you keep the metrics you just removed in 0.90_01: > buildtool_not_executable > no_dot_underscore_files > no_generated_files > > ...because you can just tag them with something like 'no_build'.
Subject: Re: [rt.cpan.org #87023] Need more flags on metrics, to make it easier to skip them
Date: Sat, 3 Aug 2013 10:38:16 -0700
To: Kenichi Ishigaki via RT <bug-Module-CPANTS-Analyse [...] rt.cpan.org>
From: Karen Etheridge <ether [...] cpan.org>
On Sat, Aug 03, 2013 at 12:36:14AM -0400, Kenichi Ishigaki via RT wrote: Show quoted text
> No. Removing them from MCA is better because other users (like T::K::Extra) may suffer the same issue.
These metrics still have value, e.g. on the CPANTS website. They just don't make sense to run on inflated dist directories. Ultimately you need to decide: who are the intended users of this distribution? The CPANTS site? Module authors wanting to test their dists? Or both? Some metrics are equally applicable to both, but others aren't, so if you want to continue to serve both sets of users, there needs to be a way for these users to select only the metrics that are useful to them.
On Sun Aug 04 02:38:26 2013, ETHER wrote: Show quoted text
> On Sat, Aug 03, 2013 at 12:36:14AM -0400, Kenichi Ishigaki via RT > wrote:
> > No. Removing them from MCA is better because other users (like > > T::K::Extra) may suffer the same issue.
> > These metrics still have value, e.g. on the CPANTS website. They just > don't make sense to run on inflated dist directories. > > Ultimately you need to decide: who are the intended users of this > distribution? The CPANTS site? Module authors wanting to test their > dists? > Or both? Some metrics are equally applicable to both, but others > aren't, > so if you want to continue to serve both sets of users, there needs to > be a > way for these users to select only the metrics that are useful to > them.
No need to decide. CPANTS is a testing service for uploaded (finalized) CPAN distributions, and the metrics are for the service (ie. finalized, well-formed distributions). Some of the metrics may happen to work locally, but don't expect metrics authors/contributors take local (ab)users into account and add some flags for them. Period.
Subject: Re: [rt.cpan.org #87023] Need more flags on metrics, to make it easier to skip them
Date: Sat, 3 Aug 2013 18:32:03 -0700
To: Kenichi Ishigaki via RT <bug-Module-CPANTS-Analyse [...] rt.cpan.org>
From: Karen Etheridge <ether [...] cpan.org>
On Sat, Aug 03, 2013 at 04:46:53PM -0400, Kenichi Ishigaki via RT wrote: Show quoted text
> No need to decide. CPANTS is a testing service for uploaded (finalized) CPAN distributions, and the metrics are for the service (ie. finalized, well-formed distributions). Some of the metrics may happen to work locally, but don't expect metrics authors/contributors take local (ab)users into account and add some flags for them. Period.
Do you feel Test::Kwalitee is an abuser? If so, it can simply be removed and save everyone a lot of trouble trying to make things work where they are not intended to be.
Sorry I needed to explain more carefully. As I said, CPANTS is a remote testing service for finalized distributions, and its metrics naturally expect files in the target directory are directly extracted from a finalized distribution. There shouldn't be any files for local usage (such as version control files, or temporary files), and if those files should be found, it would be a distribution's flaw. This has been the bottom line for a long time. However, we know some of the metrics may happen to work locally. These days more files are generated by authoring tools like dzil, and thus it's harder than before to say which metrcis should always work for everyone, but anyway it's reasonable people want to reuse some of the metrics for their local testing, assuming some of the files should be placed the same as a finalized distribution. Strictly speaking, this assumption is not always right, and there must be metrics that fail because of this wild assumption. But as long as they know what they are doing, and do it at their risks, it's fine, and it's nice if those metrics they choose to apply do happen to help. I'd be happy to apply fixes for local users if they are small and simple, don't impact overall performance and don't make further maintenance harder. This request is something different. What this means is, metrics authors/contributors should test their metrics if they work locally before releases, and if they don't work, metrics authors/contributors should add some flags for local users. That's too much. Besides, even we accept this and add some flags to metrics that may not work locally, users of older versions of TK (1.08-1.12 I suppose) will suffer from those new flagged metrics because of their (installed) TK don't support the flags. In other words, this makes conflict to those versions of TK, and as I said elsewhere, making conflicts is not an option. As for TK, it's fine as long as it tests only applicable and available metrics. You don't need to remove it at all. However, note that it's basically your responsibility to choose applicable metrics for your users, and not to expose uncertain metrics for them. That said, because of recent changes in TK that assumed core metrics should always be applicable locally, it's much harder to add more core metrics in MCK. So, it might possibly be reasonable to make MCK a bit more local files friendly (as long as it doesn't hurt performance). Anyway, you know, as there are much more to add (and many of them are from you :), don't try to make them harder to add. And as I said repeatedly, don't assume too much. Don't assume core metrics should be applicable to local users, or expected metrics are always available, or everything is sorted, or whatever. The more you assume, the less we can improve. On Sun Aug 04 10:32:15 2013, ETHER wrote: Show quoted text
> On Sat, Aug 03, 2013 at 04:46:53PM -0400, Kenichi Ishigaki via RT > wrote:
> > No need to decide. CPANTS is a testing service for uploaded > > (finalized) CPAN distributions, and the metrics are for the service > > (ie. finalized, well-formed distributions). Some of the metrics may > > happen to work locally, but don't expect metrics authors/contributors > > take local (ab)users into account and add some flags for them. > > Period.
> > Do you feel Test::Kwalitee is an abuser? If so, it can simply be > removed > and save everyone a lot of trouble trying to make things work where > they > are not intended to be.
Subject: Re: [rt.cpan.org #87023] Need more flags on metrics, to make it easier to skip them
Date: Sun, 4 Aug 2013 08:51:16 -0700
To: Kenichi Ishigaki via RT <bug-Module-CPANTS-Analyse [...] rt.cpan.org>
From: Karen Etheridge <ether [...] cpan.org>
Thanks for the longer response. On Sun, Aug 04, 2013 at 05:11:29AM -0400, Kenichi Ishigaki via RT wrote: Show quoted text
> This request is something different. What this means is, metrics authors/contributors should test their metrics if they work locally before releases, and if they don't work, metrics authors/contributors should add some flags for local users. That's too much.
I don't think it needs to be burdensome, if a few simple flags are invented in advance that distinguish metrics between "you can run this locally" and "no you can't". My proposal was the first attempt to do that, but I'm sure something better and simpler can be thought of, from the perspective of someone who understands the CPANTS service better. Can we try to find something that works? Show quoted text
> In other words, this makes conflict to those versions of TK, and as I said elsewhere, making conflicts is not an option.
As the maintainer of TK, I have already said that it very much *is* an option, and I even gave you the way to do it safely (to indicate to users of older versions of TK that they need to update, after installing a new MCA). You can't keep supporting all old versions of TK forever, or you'll never be able to do anything new. Earlier versions of TK hardcoded the list of metric names, and even enforced that all these metrics even had to exist (by planning that number of tests in Test::Builder). That was bad (or more charitably, short-sighted), and that's been removed now, and I DO NOT ASK NOR EXPECT that those versions of TK shall continue working, on this nor any future version of MCA. Please do not add code into the modules themselves to keep these versions working. It's much easier to put in some metadata into the MCA dist to say "you have a new MCA and old TK - that combination won't work. You MUST upgrade." I'm trying to work *with you* on this to make everything better, not throw up obstacles. Show quoted text
> As for TK, it's fine as long as it tests only applicable and available metrics. You don't need to remove it at all. However, note that it's basically your responsibility to choose applicable metrics for your users, and not to expose uncertain metrics for them.
That's what I'm trying to do. But because the metrics change over time, I thought that deciding on a few simple categories up front, and classifying all metrics with these, would be an easy way to prevent (most) cases of breakage. Show quoted text
> That said, because of recent changes in TK that assumed core metrics should always be applicable locally, it's much harder to add more core metrics in MCK. So, it might possibly be reasonable to make MCK a bit more local files friendly (as long as it doesn't hurt performance). > Anyway, you know, as there are much more to add (and many of them are from you :), don't try to make them harder to add. And as I said repeatedly, don't assume too much. Don't assume core metrics should be applicable to local users, or expected metrics are always available, or everything is sorted, or whatever. The more you assume, the less we can improve.
I don't want TK to block what you want to do with MCA, but if there are simple ways of keeping everything working, I'd like to consider those options.