Bug #39984 for Test-Harness: Preferences file

Sun Oct 12 09:39:06 2008 mschwern [...] cpan.org - Ticket created

Subject:

Preferences file

I would like to see TAP::Harness and prove use a configuration file. First looking in the cwd, then $HOME. I have three use cases in mind. You currently cannot run the Perl 6 tests using prove. I'm sitting in Patrick's Rakudo talk and he showed that. I know that you have to run "prove --exec './perl6'". It would be nice if Perl 6 could have its own .proverc which sets "exec: ./perl6" as the default so people don't need to know the magic flags. David Wheeler has been pushing for me to add special flags so MakeMaker can pass special flags to TAP::Harness so he can execute Postgres tests. I'm reluctant to add harness-specific code. Instead, his project could use a .tapharnessrc file which sets his necessary flags. Finally, users want to be able to turn on the extra features of TAP::Harness. Anything from colorization to parallel testing. With a .tapharnessrc in their home directory they can apply this globally. The cwd's .tapharnessrc overlays a user's defaults. For example... ~/.tapharnessrc: color: 1 jobs: 5 ~/project/.tapharnessrc: jobs: 1 # can't run in parallel When TAP::Harness (or prove) are used in the project directory, the resulting set of flags are: color: 1 jobs: 1 While .proverc and .tapharnessrc seem to be redundant, it allows the user to configure at their level of detail and understanding. Most people know prove, and that's the friendliest interface. More detailed needs, or ones that need to work in any TAP::Harness application (such as MakeMaker and Module::Build and Test::Harness) use .tapharnessrc.

Thu Oct 16 22:17:07 2008 dwheeler [...] cpan.org - Correspondence added

On Sun Oct 12 09:39:06 2008, MSCHWERN wrote: Show quoted text

> David Wheeler has been pushing for me to add special flags so MakeMaker > can pass special flags to TAP::Harness so he can execute Postgres tests. > I'm reluctant to add harness-specific code. Instead, his project could > use a .tapharnessrc file which sets his necessary flags.

Currently, to get the behavior I want, I need to specify a code reference. So either it would need to be changed so it didn't have to be a code reference, or there'd have to be support for code in the config file. Ew. —David

Thu Oct 16 22:17:09 2008 The RT System itself - Status changed from 'new' to 'open'

Thu Oct 16 22:19:15 2008 dwheeler [...] cpan.org - Cc DWHEELER added

Thu Oct 16 22:39:36 2008 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #39984] Preferences file
Date:	Thu, 16 Oct 2008 19:39:09 -0700
To:	bug-Test-Harness [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

David Wheeler via RT wrote: Show quoted text

>> David Wheeler has been pushing for me to add special flags so MakeMaker >> can pass special flags to TAP::Harness so he can execute Postgres tests. >> I'm reluctant to add harness-specific code. Instead, his project could >> use a .tapharnessrc file which sets his necessary flags.

> > Currently, to get the behavior I want, I need to specify a code > reference. So either it would need to be changed so it didn't have to be > a code reference, or there'd have to be support for code in the config > file. Ew.

Does it have to be a code ref? Could you stick the helper code into a file and then in your/project/.tapharnessrc just specify that file like so: exec: t/pgtap_helper -- Don't try the paranormal until you know what's normal. -- "Lords and Ladies" by Terry Prachett

Fri Oct 17 11:27:54 2008 dwheeler [...] cpan.org - Correspondence added

CC:	mschwern [...] cpan.org
Subject:	Re: [rt.cpan.org #39984] Preferences file
Date:	Fri, 17 Oct 2008 08:27:39 -0700
To:	bug-Test-Harness [...] rt.cpan.org
From:	"David E. Wheeler" <dwheeler [...] cpan.org>

On Oct 16, 2008, at 19:39, Michael G Schwern via RT wrote: Show quoted text

> Does it have to be a code ref? Could you stick the helper code into > a file > and then in your/project/.tapharnessrc just specify that file like so: > > exec: t/pgtap_helper

I suppose, but then I have to include *2* extra files in my distribution. This is no way to be properly lazy. In order to push adoption of pgTAP and other TAP emitting frameworks in Perl distributions, it should be *as simple as possible* to tell TAP::Harness how to test things. Ideally, I'd be able to just do it in Build.PL or Makefile.PL somehow like this: suffix_map => { t => undef, # perl r => [qw(ruby -w)], # ruby s => [qw( #pgtap psql --dbname try --username postgres --quiet --no-psqlrc --no-align --tuples-only --set ON_ERROR_ROLLBACK=1 --set ON_ERROR_STOP=1 --file %s )], }, Simple, simple, simple. Please, don't make me work for this. Best, David

Fri Mar 06 18:26:50 2009 andy [...] hexten.net - Correspondence added

I agree with the general principal and it offers a way to satisfy Petdance's requirement to run a mixture of PHP and Perl tests in the same run. But I'm concerned about the potential for a hidden config file to alter the behaviour of the harness in hard to diagnose ways. How would we feel about a single, visible config file that can only be in $PWD?

Fri Mar 06 18:47:48 2009 dwheeler [...] cpan.org - Correspondence added

CC:	mschwern [...] cpan.org
Subject:	Re: [rt.cpan.org #39984] Preferences file
Date:	Fri, 6 Mar 2009 15:47:35 -0800
To:	bug-Test-Harness [...] rt.cpan.org
From:	"David E. Wheeler" <dwheeler [...] cpan.org>

On Mar 6, 2009, at 3:26 PM, Andy Armstrong via RT wrote: Show quoted text

> How would we feel about a single, visible config file that can only > be in $PWD?

That solves one part of my problem, though I don't see why $HOME shouldn't also be allowed. But I still don't think I could configure the code I need in this way. —Theory