Skip Menu |

This queue is for tickets about the local-lib CPAN distribution.

Report information
The Basics
Id: 39644
Status: resolved
Priority: 0/
Queue: local-lib

People
Owner: apeiron [...] cpan.org
Requestors: MARKSTOS [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Wishlist
Broken in: (no value)
Fixed in: 1.002000



Subject: PATCH: add "--self-contained" option for generating self-contained dependency chains
Matt, Thanks for 'local::lib'. I've found it rather useful. I wanted to use it to generate a self-contained folder of a module with all of it's dependencies. 'local::lib' is great for helping with this, but because it keeps the global "INC" paths, it wasn't quit working for me-- dependencies already installed locally weren't added to my local directory. To solve this I created the attached patch to add a "--self-contained" option. When this flag is used, only the path to core modules and the 'local' directories are preserved. It worked well for me in testing. Mark
Subject: self_contained_local_lib.patch
--- /usr/local/share/perl/5.8.8/local/lib.pm 2008-08-06 06:52:36.000000000 -0400 +++ local/lib.pm 2008-09-27 16:38:58.000000000 -0400 @@ -14,9 +14,24 @@ our $VERSION = '1.002000'; # 1.2.0 sub import { - my ($class, $path) = @_; + my ($class, @args) = @_; + + # The path is required, but last in the list, so we pop, not shift here. + my $path = pop @args; $path = $class->resolve_path($path); $class->setup_local_lib_for($path); + + # Handle the '--self-contained' option + my $flag = shift @args; + no warnings 'uninitialized'; # the flag is optional + if ($flag eq '--self-contained') { + # The only directories that remain are those that we just defined and those where core modules are stored. + @INC = ($Config::Config{privlibexp}, $Config::Config{archlibexp}, split ':', $ENV{PERL5LIB}); + } + elsif (defined $flag) { + die "unrecognized import argument: $flag"; + } + } sub pipeline; @@ -306,6 +321,13 @@ From the shell - + # Install LWP and it's missing dependencies to the 'my_lwp' directory + perl -MCPAN -Mlocal::lib=my_lwp -e 'install LWP' + + # Install LWP and *all non-core* dependencies to the 'my_lwp' directory + perl -MCPAN -Mlocal::lib=--self-contained,my_lwp -e 'install LWP' + + # Just print out useful shell commands $ perl -Mlocal::lib export MODULEBUILDRC=/home/username/perl/.modulebuildrc export PERL_MM_OPT='INSTALL_BASE=/home/username/perl' @@ -403,6 +425,8 @@ Patches to correctly output commands for csh style shells, as well as some documentation additions, contributed by Christopher Nehren <apeiron@cpan.org>. +'--self-contained' feature contributed by Mark Stosberg <mark@summersault.com>. + =head1 LICENSE This library is free software under the same license as perl itself
Subject: Re: [rt.cpan.org #39644] PATCH: add "--self-contained" option for generating self-contained dependency chains
Date: Sun, 28 Sep 2008 14:59:55 +0100
To: MARKSTOS via RT <bug-local-lib [...] rt.cpan.org>
From: Matt S Trout <mst [...] shadowcat.co.uk>
On Sat, Sep 27, 2008 at 05:11:43PM -0400, MARKSTOS via RT wrote: Show quoted text
> Sat Sep 27 17:11:41 2008: Request 39644 was acted upon. > Transaction: Ticket created by MARKSTOS > Queue: local-lib > Subject: PATCH: add "--self-contained" option for generating > self-contained dependency chains > Broken in: (no value) > Severity: Wishlist > Owner: Nobody > Requestors: MARKSTOS@cpan.org > Status: new > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=39644 > > > > Matt, > > Thanks for 'local::lib'. I've found it rather useful. > > I wanted to use it to generate a self-contained folder of a module with > all of it's dependencies. > > 'local::lib' is great for helping with this, but because it keeps the > global "INC" paths, it wasn't quit working for me-- dependencies already > installed locally weren't added to my local directory.
Yeah, I tend to use a clean build system for that. Show quoted text
> To solve this I created the attached patch to add a "--self-contained" > option. When this flag is used, only the path to core modules and the > 'local' directories are preserved.
Looks nice, but could you expand the documentation to note that if anybody's upgraded a dual-life module with UNINST=1 then this will explode? I considered adding the feature but found that particular problem hit me rather often on machines that had been in use for a while - I don't mind having the feature but the caveats need to be clearly and explicitly documented. -- Matt S Trout Need help with your Catalyst or DBIx::Class project? Technical Director http://www.shadowcat.co.uk/catalyst/ Shadowcat Systems Ltd. Want a managed development or deployment platform? http://chainsawblues.vox.com/ http://www.shadowcat.co.uk/servers/
Show quoted text
> > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=39644 > > > > > Matt, > > > > Thanks for 'local::lib'. I've found it rather useful. > > > > I wanted to use it to generate a self-contained folder of a module > > with all of it's dependencies. > > > > 'local::lib' is great for helping with this, but because it keeps > > the global "INC" paths, it wasn't quit working for me-- dependencies > > already installed locally weren't added to my local directory.
> > Yeah, I tend to use a clean build system for that. >
> > To solve this I created the attached patch to add a > > "--self-contained" option. When this flag is used, only the path to > > core modules and the 'local' directories are preserved.
> > Looks nice, but could you expand the documentation to note that if > anybody's upgraded a dual-life module with UNINST=1 then this will > explode?
I want to make sure I understand this problem. Let's the module involved is CGI.pm. As understand the effect of "UNINST=1" in this case, it would cause the core version to be removed, correct? With that assumption, here a couple of cases spelled out. Let's say that you've installed a newer version of CGI.pm in location "a" and removed the version in the core. You now want to create a bundle of CGI.pm and all it's dependencies, in directory "b". There should be no conflict here, because "--self-contained" would only look in "b" and the core. It would find CGI.pm was missing completely, and bring down it, and it's dependencies. ### What's an example of a case where things "blow up"... what goes wrong? Mark
CC: undisclosed-recipients: ;
Subject: Re: [rt.cpan.org #39644] PATCH: add "--self-contained" option for generating self-contained dependency chains
Date: Wed, 12 Nov 2008 19:23:29 +0000
To: MARKSTOS via RT <bug-local-lib [...] rt.cpan.org>
From: Matt S Trout <mst [...] shadowcat.co.uk>
On Tue, Oct 21, 2008 at 11:03:16AM -0400, MARKSTOS via RT wrote: Show quoted text
> I want to make sure I understand this problem. Let's the module involved > is CGI.pm. As understand the effect of "UNINST=1" in this case, it would > cause the core version to be removed, correct? With that assumption, > here a couple of cases spelled out. > > Let's say that you've installed a newer version of CGI.pm in location > "a" and removed the version in the core. > > You now want to create a bundle of CGI.pm and all it's dependencies, in > directory "b". > > There should be no conflict here, because "--self-contained" would only > look in "b" and the core. It would find CGI.pm was missing completely, > and bring down it, and it's dependencies. > > ### > > What's an example of a case where things "blow up"... what goes wrong?
Now what happens when code on the same system that uses CGI.pm tries to run? *BOOM* This is especially upsetting when the module that UNINST=1 removed is something that CPAN.pm needs to run ... -- Matt S Trout Need help with your Catalyst or DBIx::Class project? Technical Director http://www.shadowcat.co.uk/catalyst/ Shadowcat Systems Ltd. Want a managed development or deployment platform? http://chainsawblues.vox.com/ http://www.shadowcat.co.uk/servers/
On Wed Nov 12 14:24:13 2008, mst@shadowcat.co.uk wrote: Show quoted text
> On Tue, Oct 21, 2008 at 11:03:16AM -0400, MARKSTOS via RT wrote:
> > I want to make sure I understand this problem. Let's say the module
involved Show quoted text
> > is CGI.pm. As I understand the effect of "UNINST=1" in this case, it
would Show quoted text
> > cause the core version to be removed, correct? With that assumption, > > here a couple of cases spelled out. > > > > Let's say that you've installed a newer version of CGI.pm in location > > "a" and removed the version in the core. > > > > You now want to create a bundle of CGI.pm and all it's dependencies, in > > directory "b". > > > > There should be no conflict here, because "--self-contained" would only > > look in "b" and the core. It would find CGI.pm was missing completely, > > and bring down it, and it's dependencies. > > > > ### > > > > What's an example of a case where things "blow up"... what goes wrong?
> > Now what happens when code on the same system that uses CGI.pm tries > to run? > > *BOOM* > > This is especially upsetting when the module that UNINST=1 removed is > something > that CPAN.pm needs to run ...
Thanks for the reply, Matt. Here's my refined understanding of the problem: - A user users "local::lib" in conjunction with "make UNINST=1" to try to do a "clean install" of a module in a private directory. - Unintuitively and without prompting, UNINST now deletes modules from the core, although this is different than the installation target. - Now a user tries a module that depends on one that was deleted from the core, but now the old version is missing from there and the new version is installed in some local place. Is this the *BOOM* scenerio that concerns you? If "UNINST=1" does indeed delete modules in a place other than the one it is going to install into, I consider that a bug with the system executiing that logic (ExtUtils::MakeMaker?). It should not allow that, or at least it should have a warning or confirmation before proceeding. "local::lib" works on INC path manipulation. It does not deal in deletion policies. I don't see that a possible bug with UNINST=1 diminishes the usefulness of "--self-contained" patch for reasonable users. If it remains a concern a note can be added to the local::lib documentation: "Note that using local::lib in combination with 'make UNINST=1' can produce unexpected results, like deleting modules in one directory and recreating them in another. Use them together with caution.". Mark
Here's a revised patch that adds a warning clause refactor as you see fit.
--- /usr/local/share/perl/5.8.8/local/lib.pm 2008-08-06 06:52:36.000000000 -0400 +++ lib.pm 2008-12-04 20:40:41.000000000 -0500 @@ -14,9 +14,24 @@ our $VERSION = '1.002000'; # 1.2.0 sub import { - my ($class, $path) = @_; + my ($class, @args) = @_; + + # The path is required, but last in the list, so we pop, not shift here. + my $path = pop @args; $path = $class->resolve_path($path); $class->setup_local_lib_for($path); + + # Handle the '--self-contained' option + my $flag = shift @args; + no warnings 'uninitialized'; # the flag is optional + if ($flag eq '--self-contained') { + # The only directories that remain are those that we just defined and those where core modules are stored. + @INC = ($Config::Config{privlibexp}, $Config::Config{archlibexp}, split ':', $ENV{PERL5LIB}); + } + elsif (defined $flag) { + die "unrecognized import argument: $flag"; + } + } sub pipeline; @@ -306,6 +321,13 @@ From the shell - + # Install LWP and it's missing dependencies to the 'my_lwp' directory + perl -MCPAN -Mlocal::lib=my_lwp -e 'install LWP' + + # Install LWP and *all non-core* dependencies to the 'my_lwp' directory + perl -MCPAN -Mlocal::lib=--self-contained,my_lwp -e 'install LWP' + + # Just print out useful shell commands $ perl -Mlocal::lib export MODULEBUILDRC=/home/username/perl/.modulebuildrc export PERL_MM_OPT='INSTALL_BASE=/home/username/perl' @@ -365,6 +387,16 @@ These values are then available for reference by any code after import. +=head1 A WARNING ABOUT UNINST=1 + +Be careful about using local::lib in combination with "make install UNINST=1". +The idea of this feature is that will uninstall an old version of a module +before installing a new one. However it lacks a safety check that the old +version and the new version will go in the same directory. Used in combination +with local::lib, you can potentially delete a globally accessible version of a +module while installing the new version in a local place. Only combine if "make +install UNINST=1" and local::lib if you understand these possible consequences. + =head1 LIMITATIONS Rather basic shell detection. Right now anything with csh in its name is @@ -403,6 +435,8 @@ Patches to correctly output commands for csh style shells, as well as some documentation additions, contributed by Christopher Nehren <apeiron@cpan.org>. +'--self-contained' feature contributed by Mark Stosberg <mark@summersault.com>. + =head1 LICENSE This library is free software under the same license as perl itself
This has been applied in version 1.3.0. Thanks!
Subject: PATCH: more docs for "--self-contained"
Matt Trout provided me a gentle reminder that the docs for --self- contained could more comprehensive. Attached is a doc-patch to extend the documentation for that feature.
Subject: self-contained-docs.diff
--- /usr/local/share/perl/5.10.0/local/lib.pm 2009-11-07 20:27:23.000000000 -0500 +++ lib.pm 2010-02-12 13:25:27.276990466 -0500 @@ -585,6 +585,26 @@ These values are then available for reference by any code after import. +=head1 CREATING A SELF-CONTAINED SET OF MODULES + +You can use local::lib to prepare a directory which contains a module and all +of its non-core dependencies. The C<--self-contained> option ignores any +globally installed modules when resolving dependencies, only considering +modules installed in a "local::lib" directory or provided by core Perl. + +A use-case for this feature would be to prepare to deploy a whole "stack" of +module dependencies on a new machine, even if you have copies of the same +dependencies installed globally already. + +The C<--self-contained> option should be used like this: + + # Install LWP and *all non-core* dependencies to the 'my_lwp' directory + perl -MCPAN -Mlocal::lib=--self-contained,my_lwp -e 'CPAN::install(LWP)' + +Note that some dependencies may involve C-based "XS" code even if your target +module doesn't. The issue of dealing with XS vs Pure Perl code is beyond the scope +of what local::lib provides. + =head1 METHODS =head2 ensure_directory_structure_for
Applied in svn as of a few days ago. Thanks!