Bug #47396 for ExtUtils-MakeMaker: expanding $(PREFIX) in Makefile

Sat Jun 27 20:51:35 2009 user42 [...] zip.com.au - Ticket created

Subject:	expanding $(PREFIX) in Makefile
Date:	Sun, 28 Jun 2009 10:51:04 +1000
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Kevin Ryde <user42 [...] zip.com.au>

If I'm not mistaken past versions of MakeMaker generated stuff like INSTALLMAN3DIR = $(PERLPREFIX)/share/man/man3 whereas in 6.52 it's locked down as INSTALLMAN3DIR = /usr/share/man/man3 The use of $(PREFIX) in the format allowed the prefix to be overridden in later make commands. Debian "cdbs" for instance had been using make install PREFIX=/some/dir/usr This was presumably only from when DESTDIR wasn't available, but for compatibility with anyone else still doing the same it could be helpful for MakeMaker to keep the former calculated style.

Sun Jun 28 00:16:42 2009 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #47396] expanding $(PREFIX) in Makefile
Date:	Sun, 28 Jun 2009 00:16:17 -0400
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

Kevin Ryde via RT wrote: Show quoted text

> If I'm not mistaken past versions of MakeMaker generated stuff like > > INSTALLMAN3DIR = $(PERLPREFIX)/share/man/man3 > > whereas in 6.52 it's locked down as > > INSTALLMAN3DIR = /usr/share/man/man3 > > The use of $(PREFIX) in the format allowed the prefix to be overridden > in later make commands. Debian "cdbs" for instance had been using > > make install PREFIX=/some/dir/usr > > This was presumably only from when DESTDIR wasn't available, but for > compatibility with anyone else still doing the same it could be helpful > for MakeMaker to keep the former calculated style.

I hear what you're saying. In theory this could be fixed, but I'd have to plow up the prefixification code again and risk new breakage. But PREFIIX doesn't actually work. Its broken by design. You might get all sorts of funny outcomes by using PREFIX. See http://schwern.org/talks/PREFIX/ for an explanation. Particularly slides 7-10. Though it worked ok for packaging, it didn't work for anything else. So I'm disinclined to put any more work into it. Sorry. Use DESTDIR. Its been six years since it was added. Debian already has a new enough MakeMaker. -- Whip me, beat me, make my code compatible with VMS!

Sun Jun 28 00:16:42 2009 The RT System itself - Status changed from 'new' to 'open'

Tue Jun 30 19:20:20 2009 user42 [...] zip.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #47396] expanding $(PREFIX) in Makefile
Date:	Wed, 01 Jul 2009 09:18:46 +1000
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Kevin Ryde <user42 [...] zip.com.au>

"Michael G Schwern via RT" <bug-ExtUtils-MakeMaker@rt.cpan.org> writes: Show quoted text

> > But PREFIIX doesn't actually work. Its broken by design.

Well, of course it's supposed to be simply the /usr part of /usr/lib etc. It's only programs which spray stuff outside a single tree that get themselves into trouble (like perl on msdos or whatever I suppose! :-). I think overriding a prefix at the "make install" stage is fairly traditional (not just with perl), even if not a fantastically great idea. If you already detect the prefix on paths in %Config it could be well worth propagating that to the makefile.

Wed Jul 01 20:35:12 2009 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #47396] expanding $(PREFIX) in Makefile
Date:	Wed, 01 Jul 2009 19:34:49 -0500
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

Kevin Ryde via RT wrote: Show quoted text

> "Michael G Schwern via RT" <bug-ExtUtils-MakeMaker@rt.cpan.org> writes:

>> But PREFIIX doesn't actually work. Its broken by design.

> > Well, of course it's supposed to be simply the /usr part of /usr/lib > etc. It's only programs which spray stuff outside a single tree that > get themselves into trouble (like perl on msdos or whatever I suppose! > :-).

Yes, its *supposed* to be simple. But stretch that across nine years of Perl releases, three different major operating systems (Unix, Windows, VMS), countless OS variants, different filesystems, at least a half dozen flavors of make, different shells, different path syntax and vendors doing lord knows what to Perl installs and it becomes very, very, very not simple. Debian has done this in the recent past, IIRC it had to do with the /usr/lib /usr/share split, and OS X currently does this putting things in /Library and /usr/local with a siteprefix of /usr/local. Oh, don't forget to make it work on VMS. <--- not a joke Show quoted text

> I think overriding a prefix at the "make install" stage is fairly > traditional (not just with perl), even if not a fantastically great > idea. If you already detect the prefix on paths in %Config it could be > well worth propagating that to the makefile.

I agree, its very traditional. That's not much of an argument, just a statement. And it doesn't fix the problem(s). Add to the mix that Perl does not have one prefix but four. PERLPREFIX for core modules, SITEPREFIX for locally installed modules and VENDORPREFIX for vendor packaged modules. PREFIX is a fiction to simplify the problem of installing a single module. The multiple prefixes were put in place to accommodate, amongst other things, Debian's layout. So, when someone writes: perl Makefile.PL make make test make install PREFIX=/some/dir How does Makefile.PL know in advance to prefix all the INSTALL directories with PREFIX and not PERL/SITE/VENDORPREFIX? That's the essential problem. You can tell it you're going to use a single PREFIX like so: perl Makefile.PL PREFIX=/usr make make test make install PREFIX=/some/other/dir And then it will know and that will work. Its complicated and it sucks and its way too late to change. The essential problem is trying to encode a dynamic situation into a static file (the Makefile) which is why we made Module::Build. -- The past has a vote, but not a veto. -- Mordecai M. Kaplan

Tue Jul 07 18:34:56 2009 user42 [...] zip.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #47396] expanding $(PREFIX) in Makefile
Date:	Wed, 08 Jul 2009 08:34:23 +1000
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Kevin Ryde <user42 [...] zip.com.au>

"Michael G Schwern via RT" <bug-ExtUtils-MakeMaker@rt.cpan.org> writes: Show quoted text

> > perl Makefile.PL > make > make test > make install PREFIX=/some/dir > > How does Makefile.PL know in advance to prefix all the INSTALL directories > with PREFIX and not PERL/SITE/VENDORPREFIX? That's the essential problem.

Oh, err, umm, yeah. You'd be tempted to have a single set of install dirs just using PREFIX, then default PREFIX from among perl/site/vendor. I suppose what you say about the structure or the "../../Library" bits being inconsistent among the three defeats that :-(. Show quoted text

> perl Makefile.PL PREFIX=/usr > make > make test > make install PREFIX=/some/other/dir > > And then it will know and that will work.

Perhaps that's close enough. (Personally I've always found the perl/site/vendor distinction and the triplicated set of directories to be fairly impenetrable :-), beyond that only 1 of the 3 at various times did something like I thought I wanted!)

Tue Jul 07 19:24:26 2009 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #47396] expanding $(PREFIX) in Makefile
Date:	Tue, 07 Jul 2009 16:23:51 -0700
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

Kevin Ryde via RT wrote: Show quoted text

>> perl Makefile.PL PREFIX=/usr >> make >> make test >> make install PREFIX=/some/other/dir >> >> And then it will know and that will work.

> > Perhaps that's close enough. > > (Personally I've always found the perl/site/vendor distinction and the > triplicated set of directories to be fairly impenetrable :-), beyond > that only 1 of the 3 at various times did something like I thought I > wanted!)

Part of the problem is the whole philosophy is never fully explained. Part of it is that until 5.12 the default look up order is wrong. Its pretty simple. Think of it like MacPorts or fink which doesn't control the entire operating system. You effectively have three managers working at the same time. You have the user installing things by hand. You have the package manager. And then there's the operating system. In a flat layout they'll stomp all over each other. You might install an upgraded program from MacPorts and then have the next OS upgrade downgrade it again. The user might install the latest version from a tarball and then MacPorts installs an older version on top of it next upgrade. "site" is what gets installed by the user, like /usr/local. "vendor" is what gets installed by the package manager. /opt/local for MacPorts, /sw for fink. "core" is what originally shipped with the operating system (or in Perl's case with Perl itself), like /usr. You look in /usr/local first, then /opt/local, then /usr. In fact, that's how my PATH is set up. Unfortunately Perl itself incorrectly puts "core" first. Fortunately Debian (and I think Redhat now) fixes that. And 5.12 will finally fix it. Packagers should be setting INSTALLDIRS=vendor. The CPAN shell should be using INSTALLDIRS=site. Nothing should be using perl/core. The broken lookup order complicates this because upgrades to dual-life CPAN modules have to go into core else they're not found. -- You know what the chain of command is? It's the chain I go get and beat you with 'til you understand who's in ruttin' command here. -- Jayne Cobb, "Firefly"

Mon Jul 13 21:25:53 2009 user42 [...] zip.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #47396] expanding $(PREFIX) in Makefile
Date:	Tue, 14 Jul 2009 11:25:19 +1000
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Kevin Ryde <user42 [...] zip.com.au>

"Michael G Schwern via RT" <bug-ExtUtils-MakeMaker@rt.cpan.org> writes: Show quoted text

> > "site" is what gets installed by the user, like /usr/local. > > "vendor" is what gets installed by the package manager. /opt/local for > MacPorts, /sw for fink.

Ah, well those two are probably the confusing bits. As an ordinary user I'd probably be happy not to know, initially, there was a "vendor" directories choice :-) Show quoted text

> Unfortunately Perl itself incorrectly puts "core" first. > Fortunately Debian (and I think Redhat now) fixes that. > And 5.12 will finally fix it.

I suppose if it had been the right way around there'd have been very little call for INSTALLDIRS=core, which would have just about wound it right back to "site" alone (with "vendor" semi-hidden away), driven off a default or specified $prefix.

Tue Jul 14 01:11:25 2009 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #47396] expanding $(PREFIX) in Makefile
Date:	Mon, 13 Jul 2009 22:11:02 -0700
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

Kevin Ryde via RT wrote: Show quoted text

> "Michael G Schwern via RT" <bug-ExtUtils-MakeMaker@rt.cpan.org> writes:

>> "site" is what gets installed by the user, like /usr/local. >> >> "vendor" is what gets installed by the package manager. /opt/local for >> MacPorts, /sw for fink.

> > Ah, well those two are probably the confusing bits. As an ordinary user > I'd probably be happy not to know, initially, there was a "vendor" > directories choice :-)

Yes. Perl, by default, doesn't have a "vendor prefix" at all leaving it effectively just "site" and "core". Show quoted text

>> Unfortunately Perl itself incorrectly puts "core" first. >> Fortunately Debian (and I think Redhat now) fixes that. >> And 5.12 will finally fix it.

> > I suppose if it had been the right way around there'd have been very > little call for INSTALLDIRS=core, which would have just about wound it > right back to "site" alone (with "vendor" semi-hidden away), driven off > a default or specified $prefix.

Exactly. Joe User would just use INSTALLDIRS=site, the CPAN shell would default to it. The "core" library files shipped with Perl would never change, new versions from CPAN would overlay them in "site". Vendors would hard code "INSTALLDIRS=vendor" in their package generators and be done with it. -- If at first you don't succeed--you fail. -- "Portal" demo

Tue Sep 15 05:45:51 2009 ntyni [...] iki.fi - Correspondence added

On Sat Jun 27 20:51:35 2009, user42@zip.com.au wrote: Show quoted text

> If I'm not mistaken past versions of MakeMaker generated stuff like > > INSTALLMAN3DIR = $(PERLPREFIX)/share/man/man3 > > whereas in 6.52 it's locked down as > > INSTALLMAN3DIR = /usr/share/man/man3

FYI, http://bugs.debian.org/545904 is about this. Turns out we've been carrying a Debian-specific patch to ExtUtils::MM_Unix::prefixify since 2001 or so to enable overriding PREFIX at install time. I assume it worked for Kevin until now because of this. I accidentally dropped the patch in my preliminary 5.10.1 packages, which made about 250 packages in the Debian archive fail to build in my tests. I think we should get rid of the Debian patch and change all our packages to use DESTDIR, but that's going to take some time, so I'll reinstate it for now and start pushing a transition. I'm attaching a current version of the patch for reference. Michael, many thanks for your work. Much appreciated. -- Niko Tyni ntyni@debian.org

From 5d783c396b6900ae049a3d0c4af1099813dcb222 Mon Sep 17 00:00:00 2001 From: Niko Tyni <ntyni@debian.org> Date: Tue, 15 Sep 2009 08:12:18 +0300 Subject: [PATCH] reinstate Debian change to ExtUtils::MakeMaker for now to allow overriding PREFIX at installation time again. (Closes: #545904) --- lib/ExtUtils/MM_Unix.pm | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/lib/ExtUtils/MM_Unix.pm b/lib/ExtUtils/MM_Unix.pm index 33a27ff..c3504b6 100644 --- a/lib/ExtUtils/MM_Unix.pm +++ b/lib/ExtUtils/MM_Unix.pm @@ -2963,8 +2963,7 @@ sub prefixify { print STDERR " prefixify $var => $path\n" if $Verbose >= 2; print STDERR " from $sprefix to $rprefix\n" if $Verbose >= 2; - if( $self->{ARGS}{PREFIX} && - $path !~ s{^\Q$sprefix\E\b}{$rprefix}s ) + if( $path !~ s{^\Q$sprefix\E\b}{$rprefix}s && $self->{ARGS}{PREFIX} ) { print STDERR " cannot prefix, using default.\n" if $Verbose >= 2; -- 1.6.3.3

Thu Sep 17 04:33:58 2009 dmacks [...] netspace.org - Correspondence added

Subject:	Re: [rt.cpan.org #47396] expanding $(PREFIX) in Makefile
Date:	Thu, 17 Sep 2009 03:56:58 -0400
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Daniel Macks <dmacks [...] netspace.org>

What was the earliest released version that contains support for DESTDIR? I manage systems with perl back to 5.8.1, which corelist says comes with ExtUtils::MakeMaker 6.17. Presently I have build scripts that use the PREFIX approach, easy enough for me to convert them to the modern DESTDIR approach. However, will I be able to use a single build script even for systems that old, or do I need to keep a legacy set of PREFIX-based build scripts for some of my older perls? dan -- Daniel Macks dmacks@netspace.org http://www.netspace.org/~dmacks

Thu Sep 17 23:45:34 2009 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #47396] expanding $(PREFIX) in Makefile
Date:	Thu, 17 Sep 2009 20:44:43 -0700
To:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

dmacks@netspace.org via RT wrote: Show quoted text

> What was the earliest released version that contains support for > DESTDIR? I manage systems with perl back to 5.8.1, which corelist says > comes with ExtUtils::MakeMaker 6.17. Presently I have build scripts > that use the PREFIX approach, easy enough for me to convert them to > the modern DESTDIR approach. However, will I be able to use a single > build script even for systems that old, or do I need to keep a legacy > set of PREFIX-based build scripts for some of my older perls?

A quick look through the public Changes file shows 6.06_01 with some DESTDIR related bug fixes in 6.10_08. So 6.17 is fine. And, of course, if for some reason it won't work with those older versions, ExtUtils::MakeMaker is updatable from CPAN. -- ...they shared one last kiss that left a bitter yet sweet taste in her mouth--kind of like throwing up after eating a junior mint. -- Dishonorable Mention, 2005 Bulwer-Lytton Fiction Contest by Tami Farmer

Sun Sep 20 21:10:46 2009 user42 [...] zip.com.au - Correspondence added

CC:	bug-ExtUtils-MakeMaker [...] rt.cpan.org
Subject:	Re: [rt.cpan.org #47396] expanding $(PREFIX) in Makefile
Date:	Mon, 21 Sep 2009 11:09:54 +1000
To:	dmacks [...] netspace.org
From:	Kevin Ryde <user42 [...] zip.com.au>

"dmacks@netspace.org via RT" <bug-ExtUtils-MakeMaker@rt.cpan.org> writes: Show quoted text

> > What was the earliest released version that contains support for > DESTDIR? I manage systems with perl back to 5.8.1,

5.8 was the guess for cdbs, I think, and far enough back to go for it.

Fri Nov 20 00:43:09 2009 mschwern [...] cpan.org - Correspondence added

Closing this up, PREFIX is beyond saving.

Fri Nov 20 00:43:11 2009 mschwern [...] cpan.org - Status changed from 'open' to 'rejected'