Bug #113685 for B-Hooks-EndOfScope: Presence of random 0-returning program named 'cc' in PATH adds hard dep on Variable::Magic

Mon Apr 11 12:52:52 2016 ribasushi [...] leporine.io - Ticket created

Subject:

Presence of random 0-returning program named 'cc' in PATH adds hard dep on Variable::Magic

This is an issue just like https://rt.cpan.org/Ticket/Display.html?id=113682, except in your case the optional xs prereq is injected. Attached a complete session log. Contact leont for advice how to use ExtUtils::HasCompiler (which does not have this problem).

Subject:

bheos_argh.txt

rabbit@Ahasver:~$ cpanm --look --dev B::Hooks::EndOfScope --> Working on B::Hooks::EndOfScope Fetching http://www.cpan.org/authors/id/E/ET/ETHER/B-Hooks-EndOfScope-0.16-TRIAL.tar.gz ... OK Entering /home/rabbit/.cpanm/work/1460393220.24298/B-Hooks-EndOfScope-0.16 with /bin/bash rabbit@Ahasver:~/.cpanm/work/1460393220.24298/B-Hooks-EndOfScope-0.16$ ln -sf /bin/true ~/bin/cc rabbit@Ahasver:~/.cpanm/work/1460393220.24298/B-Hooks-EndOfScope-0.16$ ls -l $(which cc) lrwxrwxrwx 1 rabbit rabbit 9 Apr 11 18:47 /home/rabbit/bin/cc -> /bin/true rabbit@Ahasver:~/.cpanm/work/1460393220.24298/B-Hooks-EndOfScope-0.16$ perl Makefile.PL Checking if your kit is complete... Looks good Generating a Unix-style Makefile Writing Makefile for B::Hooks::EndOfScope Writing MYMETA.yml and MYMETA.json rabbit@Ahasver:~/.cpanm/work/1460393220.24298/B-Hooks-EndOfScope-0.16$ grep 'PREREQ.*Magic' Makefile # PREREQ_PM => { warnings=>q[0], File::Glob=>q[0], ExtUtils::MakeMaker=>q[0], File::Spec=>q[0], strict=>q[0], Test::More=>q[0.89], Sub::Exporter::Progressive=>q[0.001006], IPC::Open2=>q[0], Module::Implementation=>q[0.05], Variable::Magic=>q[0.48] } rabbit@Ahasver:~/.cpanm/work/1460393220.24298/B-Hooks-EndOfScope-0.16$ ln -sf /bin/false ~/bin/cc rabbit@Ahasver:~/.cpanm/work/1460393220.24298/B-Hooks-EndOfScope-0.16$ ls -l $(which cc) lrwxrwxrwx 1 rabbit rabbit 10 Apr 11 18:48 /home/rabbit/bin/cc -> /bin/false rabbit@Ahasver:~/.cpanm/work/1460393220.24298/B-Hooks-EndOfScope-0.16$ perl Makefile.PL Generating a Unix-style Makefile Writing Makefile for B::Hooks::EndOfScope Writing MYMETA.yml and MYMETA.json rabbit@Ahasver:~/.cpanm/work/1460393220.24298/B-Hooks-EndOfScope-0.16$ grep 'PREREQ.*Magic' Makefile

Mon Apr 11 13:05:20 2016 ether [...] cpan.org - Correspondence added

On 2016-04-11 09:52:52, RIBASUSHI wrote: Show quoted text

> This is an issue just like > https://rt.cpan.org/Ticket/Display.html?id=113682, except in your case > the optional xs prereq is injected. > > Attached a complete session log. > > Contact leont for advice how to use ExtUtils::HasCompiler (which does > not have this problem).

Note to self: this is actually an issue in Dist::Zilla::Plugin::DynamicPrereqs, which is where the can_xs() definition is coming from -- but dates back to B::Hooks::EndOfScope again because the plugin used this module's implementation of can_xs() (so as to not cause a change in behaviour when moving to using the plugin for Makefile.PL modification). I've been monitoring the development of ExtUtils::HasCompiler, and waiting for the right time to switch to using it internally. Perhaps it is now mature enough to consider switching.

Mon Apr 11 13:05:20 2016 The RT System itself - Status changed from 'new' to 'open'

Mon Apr 18 22:21:21 2016 ether [...] cpan.org - Taken

Mon Apr 18 22:39:24 2016 ether [...] cpan.org - Correspondence added

On 2016-04-11 09:52:52, RIBASUSHI wrote: Show quoted text

> This is an issue just like > https://rt.cpan.org/Ticket/Display.html?id=113682, except in your case > the optional xs prereq is injected.

Looking at the referenced ticket, you say "As a bonus point EU::CB does not get tripped by this" -- am I correct then in inferring that your perl and EUMM are too old to understand configure-requires prereqs? I think this scenario can only happen if ExtUtils::CBuilder is not installed (which only happens if configure prereqs aren't satisfied): https://metacpan.org/source/ETHER/B-Hooks-EndOfScope-0.16-TRIAL/Makefile.PL#L117-120 This seems to be the path where we check if the cc executable is runnable, but we don't actually run it. It's the other path that uses EUCB, which you say is fine.

Mon Apr 18 22:40:44 2016 ether [...] cpan.org - Correspondence added

On 2016-04-18 19:39:24, ETHER wrote: Show quoted text

> On 2016-04-11 09:52:52, RIBASUSHI wrote:

> > This is an issue just like > > https://rt.cpan.org/Ticket/Display.html?id=113682, except in your > > case > > the optional xs prereq is injected.

> > Looking at the referenced ticket, you say "As a bonus point EU::CB > does not get tripped by this" -- > am I correct then in inferring that your perl and EUMM are too old to > understand configure-requires > prereqs? I think this scenario can only happen if ExtUtils::CBuilder > is not installed (which only > happens if configure prereqs aren't satisfied): > > https://metacpan.org/source/ETHER/B-Hooks-EndOfScope-0.16- > TRIAL/Makefile.PL#L117-120 > > This seems to be the path where we check if the cc executable is > runnable, but we don't actually run it. It's the other path that uses > EUCB, which you say is fine.

I've made a note of that here: https://github.com/karenetheridge/Dist-Zilla-Plugin-DynamicPrereqs/commit/09424a020f879b4379130b2a9b47d61428921153 However I will also pursue fixing this by using ExtUtils::HasCompiler instead.

Tue Apr 19 00:05:35 2016 ribasushi [...] leporine.io - Correspondence added

Subject:	Re: [rt.cpan.org #113685] Presence of random 0-returning program named 'cc' in PATH adds hard dep on Variable::Magic
Date:	Tue, 19 Apr 2016 06:05:17 +0200
To:	bug-B-Hooks-EndOfScope [...] rt.cpan.org
From:	Peter Rabbitson <ribasushi [...] cpan.org>

On 04/19/2016 04:39 AM, Karen Etheridge via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=113685 > > > On 2016-04-11 09:52:52, RIBASUSHI wrote:

>> This is an issue just like >> https://rt.cpan.org/Ticket/Display.html?id=113682, except in your case >> the optional xs prereq is injected.

> Looking at the referenced ticket, you say "As a bonus point EU::CB does not get tripped by this" -- > am I correct then in inferring that your perl and EUMM are too old to understand configure-requires > prereqs? I think this scenario can only happen if ExtUtils::CBuilder is not installed (which only > happens if configure prereqs aren't satisfied):

I don't follow. RT#113682 does not mention EU::CB in any manner. Perhaps you are confusing this with the completely unrelated RT#113635 ?

Tue Apr 19 00:17:58 2016 ether [...] cpan.org - Correspondence added

On 2016-04-18 21:05:35, RIBASUSHI wrote: Show quoted text

> On 04/19/2016 04:39 AM, Karen Etheridge via RT wrote:

> > <URL: https://rt.cpan.org/Ticket/Display.html?id=113685 > > > > > On 2016-04-11 09:52:52, RIBASUSHI wrote:

> >> This is an issue just like > >> https://rt.cpan.org/Ticket/Display.html?id=113682, except in your > >> case > >> the optional xs prereq is injected.

> > Looking at the referenced ticket, you say "As a bonus point EU::CB > > does not get tripped by this" -- > > am I correct then in inferring that your perl and EUMM are too old to > > understand configure-requires > > prereqs? I think this scenario can only happen if ExtUtils::CBuilder > > is not installed (which only > > happens if configure prereqs aren't satisfied):

> > I don't follow. RT#113682 does not mention EU::CB in any manner. > Perhaps you are confusing this with the completely unrelated RT#113635 > ?

Ok, so is this a problem with ExtUtils::CBuilder? That's what does the final check in can_xs, that should be returning false if `cc` fails: https://metacpan.org/source/ETHER/B-Hooks-EndOfScope-0.16-TRIAL/Makefile.PL#L156-166

Tue Apr 19 00:52:47 2016 ribasushi [...] leporine.io - Correspondence added

On Tue Apr 19 06:17:58 2016, ETHER wrote: Show quoted text

> > Ok, so is this a problem with ExtUtils::CBuilder? That's what does > the final check in can_xs, that should be returning false if `cc` > fails: https://metacpan.org/source/ETHER/B-Hooks-EndOfScope-0.16- > TRIAL/Makefile.PL#L156-166

The entire raison d'etre of ExtUtils::HasCompiler (and previously unsuccessfully of Config::Autoconf) is that EU::CB is not reliable. This is beyond "common knowledge" at this point, I can not fathom how you of all people can be unclear on this. Attached is a session log replicating the failure (something you can do from your terminal in exactly the same steps).

Subject:

eli5.txt

rabbit@Ahasver:~$ cpanm --look B::Hooks::EndOfScope@0.16 --> Working on B::Hooks::EndOfScope Fetching http://www.cpan.org/authors/id/E/ET/ETHER/B-Hooks-EndOfScope-0.16-TRIAL.tar.gz ... OK Entering /home/rabbit/.cpanm/work/1461039994.11594/B-Hooks-EndOfScope-0.16 with /bin/bash rabbit@Ahasver:~/.cpanm/work/1461039994.11594/B-Hooks-EndOfScope-0.16$ ln -s /bin/true ~/bin/cc rabbit@Ahasver:~/.cpanm/work/1461039994.11594/B-Hooks-EndOfScope-0.16$ ls -l $(which cc) lrwxrwxrwx 1 rabbit rabbit 9 Apr 19 06:21 /home/rabbit/bin/cc -> /bin/true rabbit@Ahasver:~/.cpanm/work/1461039994.11594/B-Hooks-EndOfScope-0.16$ git init . Initialized empty Git repository in /home/rabbit/.cpanm/work/1461039994.11594/B-Hooks-EndOfScope-0.16/.git/ rabbit@Ahasver:~/.cpanm/work/1461039994.11594/B-Hooks-EndOfScope-0.16$ git add . rabbit@Ahasver:~/.cpanm/work/1461039994.11594/B-Hooks-EndOfScope-0.16$ nano Makefile.PL rabbit@Ahasver:~/.cpanm/work/1461039994.11594/B-Hooks-EndOfScope-0.16$ git diff diff --git a/Makefile.PL b/Makefile.PL index e4571ec..e8a605d 100644 --- a/Makefile.PL +++ b/Makefile.PL @@ -150,15 +150,18 @@ int boot_sanexs() { END_C close $FH; +use Devel::Dwarn; + + # Can the C compiler access the same headers XS does my @libs = (); my $object = undef; eval { local $^W = 0; - $object = $builder->compile( + $object = Dwarn $builder->compile( source => $tmpfile, ); - @libs = $builder->link( + @libs = Dwarn $builder->link( objects => $object, module_name => 'sanexs', ); @@ -166,12 +169,12 @@ END_C my $result = $@ ? 0 : 1; # Clean up all the build files - foreach ( $tmpfile, $object, @libs ) { - next unless defined $_; - 1 while unlink; - } +# foreach ( $tmpfile, $object, @libs ) { +# next unless defined $_; +# 1 while unlink; +# } - return $result; + return Dwarn $result; } sub is_smoker { rabbit@Ahasver:~/.cpanm/work/1461039994.11594/B-Hooks-EndOfScope-0.16$ perl Makefile.PL "compilexs-dsLOC.o" "compilexs-dsLOC.so" 1 Generating a Unix-style Makefile Writing Makefile for B::Hooks::EndOfScope Writing MYMETA.yml and MYMETA.json rabbit@Ahasver:~/.cpanm/work/1461039994.11594/B-Hooks-EndOfScope-0.16$ ls -l compilexs* -rw------- 1 rabbit rabbit 145 Apr 19 06:28 compilexs-dsLOC.c rabbit@Ahasver:~/.cpanm/work/1461039994.11594/B-Hooks-EndOfScope-0.16$ grep Variable Makefile # PREREQ_PM => { warnings=>q[0], File::Glob=>q[0], ExtUtils::MakeMaker=>q[0], File::Spec=>q[0], strict=>q[0], Test::More=>q[0.89], Sub::Exporter::Progressive=>q[0.001006], IPC::Open2=>q[0], Module::Implementation=>q[0.05], Variable::Magic=>q[0.48] } $(NOECHO) $(ECHO) ' Variable::Magic: '\''0.48'\''' >> META_new.yml $(NOECHO) $(ECHO) ' "Variable::Magic" : "0.48",' >> META_new.json $(NOECHO) $(ECHO) ' <REQUIRE NAME="Variable::Magic" VERSION="0.48" />' >> $(DISTNAME).ppd

Tue Apr 19 01:10:17 2016 ether [...] cpan.org - Correspondence added

On 2016-04-18 21:52:47, RIBASUSHI wrote: Show quoted text

> On Tue Apr 19 06:17:58 2016, ETHER wrote:

> > > > Ok, so is this a problem with ExtUtils::CBuilder? That's what does > > the final check in can_xs, that should be returning false if `cc` > > fails: https://metacpan.org/source/ETHER/B-Hooks-EndOfScope-0.16- > > TRIAL/Makefile.PL#L156-166

> > The entire raison d'etre of ExtUtils::HasCompiler (and previously > unsuccessfully of Config::Autoconf) is that EU::CB is not reliable. > This is beyond "common knowledge" at this point, I can not fathom how > you of all people can be unclear on this.

I fail to see how being insulting is at all constructive here, when I was just asking for confirmation since apparently I got two tickets mixed up.

Tue Apr 19 01:35:46 2016 ribasushi [...] leporine.io - Correspondence added

On Tue Apr 19 07:10:17 2016, ETHER wrote: Show quoted text

> > I fail to see how being insulting is at all constructive here

Apologies if you feel insulted. I am exasperated due to the direction this conversation has taken. Let me know if there are outstanding unanswered questions – I will do my best to address them only after the frustration subsides.

Tue Apr 19 01:40:37 2016 ether [...] cpan.org - Correspondence added

On 2016-04-18 22:35:46, RIBASUSHI wrote: Show quoted text

> On Tue Apr 19 07:10:17 2016, ETHER wrote:

> > > > I fail to see how being insulting is at all constructive here

> > Apologies if you feel insulted. I am exasperated due to the direction > this conversation has taken. Let me know if there are outstanding > unanswered questions – I will do my best to address them only after > the frustration subsides.

I'm trying to clarify if the problem is all in EUCB, or if there is something (realistic) that I can do in BHEOS's Makefile.PL to make things better (other than simply switching to ExtUtils::HasCompiler).

Tue Apr 19 02:11:50 2016 ribasushi [...] leporine.io - Correspondence added

Subject:	Re: [rt.cpan.org #113685] Presence of random 0-returning program named 'cc' in PATH adds hard dep on Variable::Magic
Date:	Tue, 19 Apr 2016 08:11:34 +0200
To:	bug-B-Hooks-EndOfScope [...] rt.cpan.org
From:	Peter Rabbitson <ribasushi [...] cpan.org>

On 04/19/2016 07:40 AM, Karen Etheridge via RT wrote: Show quoted text

> I'm trying to clarify if the problem is all in EUCB, or if there is > something (realistic) that I can do in BHEOS's Makefile.PL to make > things better (other than simply switching to ExtUtils::HasCompiler).

Leont is the best person to definitively answer this for you, given he has very recently worked on both EUHC and EUCB

Tue Apr 19 04:55:05 2016 LEONT [...] cpan.org - Correspondence added

On Tue Apr 19 01:40:37 2016, ETHER wrote: Show quoted text

> I'm trying to clarify if the problem is all in EUCB, or if there is > something (realistic) that I can do in BHEOS's Makefile.PL to make > things better (other than simply switching to ExtUtils::HasCompiler).

CBuilder->have_compiler is problematic. It can easily be fixed (and I am planning to do that), but given that it won't propagate to end users without using configure-requires that's not terribly helpful. Fortunately you're already not using have_compiler which makes fixing this easier. You could fix it in can_xs by loading the product of the linking. Something like «DynaLoader::dl_load_file(File::Spec->rel2abs($lib_file)) or ...» (with $lib_file being the first return value of CBuilder->link) should do the trick. HasCompiler does a bit more than that, but the loading is the most important check by far. But yeah, on the long term HasCompiler should be the easier route for this. Leon

Tue Apr 19 04:59:13 2016 LEONT [...] cpan.org - Correspondence added

On Tue Apr 19 04:55:05 2016, LEONT wrote: Show quoted text

> But yeah, on the long term HasCompiler should be the easier route for > this.

Erm, that sounds like I think HasCompiler isn't ready for prime-time, which I don't want to imply. I meant that more in the sense of "adding one line of code is an easier change, but it's not the path towards the future" Leon

Tue Apr 19 12:37:53 2016 ether [...] cpan.org - Correspondence added

On 2016-04-19 01:59:13, LEONT wrote: Show quoted text

> On Tue Apr 19 04:55:05 2016, LEONT wrote:

> > But yeah, on the long term HasCompiler should be the easier route for > > this.

> > Erm, that sounds like I think HasCompiler isn't ready for prime-time, > which I don't want to imply. I meant that more in the sense of "adding > one line of code is an easier change, but it's not the path towards > the future"

I'm working on switching DynamicPrereqs (which BHEOS uses) from EUCB to EUHC, and will either release both as trial today or while in Rugby (and will grab you for a review). Neither release will be going stable without a few people signing off on the changes (including everyone presently in this thread).

Thu Apr 21 07:27:18 2016 ether [...] cpan.org - Correspondence added

ExtUtils::HasCompiler is being used in B-Hooks-EndOfScope-0.17-TRIAL -- please let me know how this works out!

Thu Apr 21 07:27:27 2016 ether [...] cpan.org - Status changed from 'open' to 'resolved'

Thu Apr 21 09:55:55 2016 ribasushi [...] leporine.io - Correspondence added

On Thu Apr 21 13:27:18 2016, ETHER wrote: Show quoted text

> ExtUtils::HasCompiler is being used in B-Hooks-EndOfScope-0.17-TRIAL > -- please let me know how this works out!

It doesn't: https://travis-ci.org/dbsrgits/dbix-class/jobs/124750245#L610 Also when shipping trial releases please change the status to 'patched' instead of closing. Otherwise there is no mechanism to notify me when a stable version has shipped.

Thu Apr 21 09:55:57 2016 ribasushi [...] leporine.io - Status changed from 'resolved' to 'open'

Thu Apr 21 11:16:48 2016 ether [...] cpan.org - Correspondence added

On 2016-04-21 06:55:55, RIBASUSHI wrote: Show quoted text

> On Thu Apr 21 13:27:18 2016, ETHER wrote:

> > ExtUtils::HasCompiler is being used in B-Hooks-EndOfScope-0.17-TRIAL > > -- please let me know how this works out!

> > It doesn't: > > https://travis-ci.org/dbsrgits/dbix-class/jobs/124750245#L610

Ah, yes, that's due to the overly-enthusiastic inlining of modules, which results in File::* from my install (5.23.9) being inlined unnecessarily, which I already fixed in the repo when haarg commented on it. I shall do another release forthwith.

Thu Apr 21 11:23:04 2016 ether [...] cpan.org - Correspondence added

Show quoted text

> I shall do another release forthwith.

Hopefully now fixed with B-Hooks-EndOfScope-0.18-TRIAL.

Thu Apr 21 11:23:06 2016 ether [...] cpan.org - Status changed from 'open' to 'patched'

Thu Apr 21 18:03:38 2016 ether [...] cpan.org - Fixed in 0.18-TRIAL added

Fri Apr 22 04:31:50 2016 ribasushi [...] leporine.io - Correspondence added

On Thu Apr 21 17:23:04 2016, ETHER wrote: Show quoted text

>

> > I shall do another release forthwith.

> > Hopefully now fixed with B-Hooks-EndOfScope-0.18-TRIAL.

Seems to pass ok, will retest on stable to make sure.

Mon May 02 20:50:11 2016 ether [...] cpan.org - Correspondence added

On 2016-04-22 01:31:50, RIBASUSHI wrote: Show quoted text

> On Thu Apr 21 17:23:04 2016, ETHER wrote:

> >

> > > I shall do another release forthwith.

> > > > Hopefully now fixed with B-Hooks-EndOfScope-0.18-TRIAL.

> > Seems to pass ok, will retest on stable to make sure. > >

Now stable in 0.19.

Mon May 02 20:50:19 2016 ether [...] cpan.org - Status changed from 'patched' to 'resolved'

Fri May 06 10:58:44 2016 ribasushi [...] leporine.io - Correspondence added

On Tue May 03 02:50:11 2016, ETHER wrote: Show quoted text

> On 2016-04-22 01:31:50, RIBASUSHI wrote:

> > On Thu Apr 21 17:23:04 2016, ETHER wrote:

> > >

> > > > I shall do another release forthwith.

> > > > > > Hopefully now fixed with B-Hooks-EndOfScope-0.18-TRIAL.

> > > > Seems to pass ok, will retest on stable to make sure. > >

> > Now stable in 0.19.

Sadly 0.19 broke everything again, as it reverted to bundling a large ./inc/ just like 0.17 did: https://metacpan.org/diff/file?target=ETHER%2FB-Hooks-EndOfScope-0.19%2F&source=ETHER%2FB-Hooks-EndOfScope-0.18-TRIAL%2F In fact .19 now bundles even more than .17 did: https://metacpan.org/diff/file?target=ETHER%2FB-Hooks-EndOfScope-0.19%2F&source=ETHER%2FB-Hooks-EndOfScope-0.17-TRIAL%2F

Fri May 06 10:58:46 2016 ribasushi [...] leporine.io - Status changed from 'resolved' to 'open'

Fri May 06 12:43:25 2016 ether [...] cpan.org - Correspondence added

On 2016-05-06 07:58:44, RIBASUSHI wrote: Show quoted text

> On Tue May 03 02:50:11 2016, ETHER wrote:

> > On 2016-04-22 01:31:50, RIBASUSHI wrote:

> > > On Thu Apr 21 17:23:04 2016, ETHER wrote:

> > > >

> > > > > I shall do another release forthwith.

> > > > > > > > Hopefully now fixed with B-Hooks-EndOfScope-0.18-TRIAL.

> > > > > > Seems to pass ok, will retest on stable to make sure. > > >

> > > > Now stable in 0.19.

> > Sadly 0.19 broke everything again, as it reverted to bundling a large > ./inc/ just like 0.17 did: > > https://metacpan.org/diff/file?target=ETHER%2FB-Hooks-EndOfScope- > 0.19%2F&source=ETHER%2FB-Hooks-EndOfScope-0.18-TRIAL%2F > > In fact .19 now bundles even more than .17 did: > https://metacpan.org/diff/file?target=ETHER%2FB-Hooks-EndOfScope- > 0.19%2F&source=ETHER%2FB-Hooks-EndOfScope-0.17-TRIAL%2F

I've release 0.20 with the extra crud removed, and pursuing the new regression upstream. I'm extremely sorry about that; I neglected to check the tarball diff before I released because I hadn't changed anything in this distribution itself since the last trial.. but bits upstream had moved.

Fri May 06 12:43:27 2016 ether [...] cpan.org - Status changed from 'open' to 'resolved'

Fri May 06 19:50:12 2016 ether [...] cpan.org - Fixed in 0.20 added

Fri May 06 19:50:12 2016 ether [...] cpan.org - Broken in 0.17-TRIAL added

Fri May 06 19:50:12 2016 ether [...] cpan.org - Broken in 0.19 added

Fri May 06 19:50:12 2016 ether [...] cpan.org - Correspondence added

Root issue patched in https://rt.perl.org/Ticket/Display.html?id=128089

Fri May 06 19:50:25 2016 ether [...] cpan.org - Reference to https://rt.perl.org/Ticket/Display.html?id=128089 added