Bug #90335 for App-FatPacker: Module::Pluggable interoperability

Wed Nov 13 11:46:21 2013 djerius [...] cpan.org - Ticket created

Subject:

Module::Pluggable interoperability

I'm interested in fatpacking an application which uses Module::Pluggable. The application ships with a default set of plugins, but unfortunately Module::Pluggable cannot find them as it queries the filesystem for plugins, and the packed ones don't live there. I'd like to suggest that App::FatPacker insert an object into @INC which would provide a defined interface which Module::Pluggable could query to determine the list of modules packed. I'm willing to provide code, etc., if this idea seems interesting. Thanks, Diab (See also https://rt.cpan.org/Ticket/Display.html?id=90317)

Wed Nov 13 14:46:56 2013 BINGOS [...] cpan.org - Correspondence added

On Wed Nov 13 11:46:21 2013, DJERIUS wrote: Show quoted text

> I'm interested in fatpacking an application which uses > Module::Pluggable. The application ships with a default set of > plugins, but unfortunately Module::Pluggable cannot find them as it > queries the filesystem for plugins, and the packed ones don't live > there. > > I'd like to suggest that App::FatPacker insert an object into @INC > which would provide a defined interface which Module::Pluggable could > query to determine the list of modules packed. > > I'm willing to provide code, etc., if this idea seems interesting. > > Thanks, > > Diab > > (See also https://rt.cpan.org/Ticket/Display.html?id=90317)

I found I had the same problem when trying to create a fatpacked cpanp. CPANPLUS uses Module::Pluggable and the fatpacked cpanp couldn't load fatpacked CPANPLUS plugins. My efforts are in the CPANPLUS repository if anyone is interested. https://github.com/jib/cpanplus-devel/blob/master/dev-bin/fatpack.pl#L96 https://github.com/jib/cpanplus-devel/blob/master/dev-bin/Module-Pluggable.patch

Wed Nov 13 14:46:58 2013 The RT System itself - Status changed from 'new' to 'open'

Wed Nov 13 16:03:39 2013 haarg [...] haarg.org - Correspondence added

On Wed Nov 13 14:46:56 2013, BINGOS wrote: Show quoted text

> On Wed Nov 13 11:46:21 2013, DJERIUS wrote:

> > I'm interested in fatpacking an application which uses > > Module::Pluggable. The application ships with a default set of > > plugins, but unfortunately Module::Pluggable cannot find them as it > > queries the filesystem for plugins, and the packed ones don't live > > there. > > > > I'd like to suggest that App::FatPacker insert an object into @INC > > which would provide a defined interface which Module::Pluggable could > > query to determine the list of modules packed. > > > > I'm willing to provide code, etc., if this idea seems interesting. > > > > Thanks, > > > > Diab > > > > (See also https://rt.cpan.org/Ticket/Display.html?id=90317)

> > I found I had the same problem when trying to create a fatpacked > cpanp. > > CPANPLUS uses Module::Pluggable and the fatpacked cpanp couldn't load > fatpacked CPANPLUS plugins. > > My efforts are in the CPANPLUS repository if anyone is interested. > > https://github.com/jib/cpanplus-devel/blob/master/dev- > bin/fatpack.pl#L96 > > https://github.com/jib/cpanplus-devel/blob/master/dev-bin/Module- > Pluggable.patch

I don't use Module::Pluggable, but I've had to deal with the same problem. I ended up using a similar solution: https://github.com/haarg/App-BCSSH/blob/master/maint/pack#L129

Thu Nov 14 07:00:17 2013 simonw [...] cpan.org - Correspondence added

I got a prototype working by looking for the presence of %main::fatpacked but that's obviously not a great longterm solution. On Wed Nov 13 16:03:39 2013, haarg wrote: Show quoted text

> On Wed Nov 13 14:46:56 2013, BINGOS wrote:

> > On Wed Nov 13 11:46:21 2013, DJERIUS wrote:

> > > I'm interested in fatpacking an application which uses > > > Module::Pluggable. The application ships with a default set of > > > plugins, but unfortunately Module::Pluggable cannot find them as it > > > queries the filesystem for plugins, and the packed ones don't live > > > there. > > > > > > I'd like to suggest that App::FatPacker insert an object into @INC > > > which would provide a defined interface which Module::Pluggable > > > could > > > query to determine the list of modules packed. > > > > > > I'm willing to provide code, etc., if this idea seems interesting. > > > > > > Thanks, > > > > > > Diab > > > > > > (See also https://rt.cpan.org/Ticket/Display.html?id=90317)

> > > > I found I had the same problem when trying to create a fatpacked > > cpanp. > > > > CPANPLUS uses Module::Pluggable and the fatpacked cpanp couldn't load > > fatpacked CPANPLUS plugins. > > > > My efforts are in the CPANPLUS repository if anyone is interested. > > > > https://github.com/jib/cpanplus-devel/blob/master/dev- > > bin/fatpack.pl#L96 > > > > https://github.com/jib/cpanplus-devel/blob/master/dev-bin/Module- > > Pluggable.patch

> > I don't use Module::Pluggable, but I've had to deal with the same > problem. I ended up using a similar solution: > > https://github.com/haarg/App-BCSSH/blob/master/maint/pack#L129

Fri Nov 15 12:36:46 2013 djerius [...] cpan.org - Correspondence added

On Thu Nov 14 07:00:17 2013, SIMONW wrote: Show quoted text

> I got a prototype working by looking for the presence of > %main::fatpacked but that's obviously not a great longterm solution. > >

Ok, here are my takes on this: https://github.com/djerius/App-FatPacker/commit/e7051d24e8b8cc6556d3e2fcdaf00a1348e4a791 https://github.com/djerius/Module-Pluggable/commit/a92be511035f6479fe5d8ff93277060150ac8e4e It shoves an object into @INC rather than a subroutine ref. I prefer this solution as it's based upon behavior (whether the @INC hook provides a method) rather than by name. That way we can define an API and it'll be completely independent of knowing who is doing what.

Fri Nov 15 13:03:55 2013 haarg [...] haarg.org - Correspondence added

On Fri Nov 15 12:36:46 2013, DJERIUS wrote: Show quoted text

> On Thu Nov 14 07:00:17 2013, SIMONW wrote:

> > I got a prototype working by looking for the presence of > > %main::fatpacked but that's obviously not a great longterm solution. > > > >

> > Ok, here are my takes on this: > > https://github.com/djerius/App- > FatPacker/commit/e7051d24e8b8cc6556d3e2fcdaf00a1348e4a791 > > https://github.com/djerius/Module- > Pluggable/commit/a92be511035f6479fe5d8ff93277060150ac8e4e > > It shoves an object into @INC rather than a subroutine ref. > > I prefer this solution as it's based upon behavior (whether the @INC > hook provides a method) rather than by name. > > That way we can define an API and it'll be completely independent of > knowing who is doing what.

This will need some slight adjustment to work in perl 5.6. It should just be a matter of moving the version check inside the INC sub, returning the sub rather than the filehandle.

Fri Nov 15 13:51:11 2013 djerius [...] cpan.org - Correspondence added

On Fri Nov 15 13:03:55 2013, haarg wrote: Show quoted text

> On Fri Nov 15 12:36:46 2013, DJERIUS wrote:

> > On Thu Nov 14 07:00:17 2013, SIMONW wrote:

> > > I got a prototype working by looking for the presence of > > > %main::fatpacked but that's obviously not a great longterm > > > solution. > > > > > >

> > > > Ok, here are my takes on this: > > > > https://github.com/djerius/App- > > FatPacker/commit/e7051d24e8b8cc6556d3e2fcdaf00a1348e4a791 > > > > https://github.com/djerius/Module- > > Pluggable/commit/a92be511035f6479fe5d8ff93277060150ac8e4e > > > > It shoves an object into @INC rather than a subroutine ref. > > > > I prefer this solution as it's based upon behavior (whether the @INC > > hook provides a method) rather than by name. > > > > That way we can define an API and it'll be completely independent of > > knowing who is doing what.

> > This will need some slight adjustment to work in perl 5.6. It should > just be a matter of moving the version check inside the INC sub, > returning the sub rather than the filehandle.

Is there some subtlety I'm missing? We already know which version of perl is running prior to defining the INC sub; the code returns a sub if $] < 5.008, the object otherwise. So < 5.008 can never see a filehandle. If the object is in play, we already know we're >= 5.008, so it can return the filehandle. Or...?

Fri Nov 15 15:02:56 2013 haarg [...] haarg.org - Correspondence added

On Fri Nov 15 13:51:11 2013, DJERIUS wrote: Show quoted text

> On Fri Nov 15 13:03:55 2013, haarg wrote:

> > On Fri Nov 15 12:36:46 2013, DJERIUS wrote:

> > > On Thu Nov 14 07:00:17 2013, SIMONW wrote:

> > > > I got a prototype working by looking for the presence of > > > > %main::fatpacked but that's obviously not a great longterm > > > > solution. > > > > > > > >

> > > > > > Ok, here are my takes on this: > > > > > > https://github.com/djerius/App- > > > FatPacker/commit/e7051d24e8b8cc6556d3e2fcdaf00a1348e4a791 > > > > > > https://github.com/djerius/Module- > > > Pluggable/commit/a92be511035f6479fe5d8ff93277060150ac8e4e > > > > > > It shoves an object into @INC rather than a subroutine ref. > > > > > > I prefer this solution as it's based upon behavior (whether the > > > @INC > > > hook provides a method) rather than by name. > > > > > > That way we can define an API and it'll be completely independent > > > of > > > knowing who is doing what.

> > > > This will need some slight adjustment to work in perl 5.6. It should > > just be a matter of moving the version check inside the INC sub, > > returning the sub rather than the filehandle.

> > Is there some subtlety I'm missing? We already know which version of > perl is running prior to defining the INC sub; the code returns a sub > if $] < 5.008, the object otherwise. So < 5.008 can never see a > filehandle. If the object is in play, we already know we're >= 5.008, > so it can return the filehandle. > > Or...?

The 5.6 code should also use an object. The only difference should be in what is returned from the INC sub.

Fri Nov 15 15:51:10 2013 djerius [...] cpan.org - Correspondence added

On Fri Nov 15 15:02:56 2013, haarg wrote: Show quoted text

> On Fri Nov 15 13:51:11 2013, DJERIUS wrote:

> > On Fri Nov 15 13:03:55 2013, haarg wrote:

> > > On Fri Nov 15 12:36:46 2013, DJERIUS wrote:

> > > > On Thu Nov 14 07:00:17 2013, SIMONW wrote:

> > > > > I got a prototype working by looking for the presence of > > > > > %main::fatpacked but that's obviously not a great longterm > > > > > solution. > > > > > > > > > >

> > > > > > > > Ok, here are my takes on this: > > > > > > > > https://github.com/djerius/App- > > > > FatPacker/commit/e7051d24e8b8cc6556d3e2fcdaf00a1348e4a791 > > > > > > > > https://github.com/djerius/Module- > > > > Pluggable/commit/a92be511035f6479fe5d8ff93277060150ac8e4e > > > > > > > > It shoves an object into @INC rather than a subroutine ref. > > > > > > > > I prefer this solution as it's based upon behavior (whether the > > > > @INC > > > > hook provides a method) rather than by name. > > > > > > > > That way we can define an API and it'll be completely independent > > > > of > > > > knowing who is doing what.

> > > > > > This will need some slight adjustment to work in perl 5.6. It > > > should > > > just be a matter of moving the version check inside the INC sub, > > > returning the sub rather than the filehandle.

> > > > Is there some subtlety I'm missing? We already know which version of > > perl is running prior to defining the INC sub; the code returns a sub > > if $] < 5.008, the object otherwise. So < 5.008 can never see a > > filehandle. If the object is in play, we already know we're >= > > 5.008, > > so it can return the filehandle. > > > > Or...?

> > The 5.6 code should also use an object. The only difference should be > in what is returned from the INC sub.

I'm obviously not up on my Perl archeology. Digging through the docs for require I find this: 5.6: no mention of INC hooks http://search.cpan.org/~gsar/perl-5.6.1/pod/perlfunc.pod#require 5.8: hooks may be sub, array, or object. first level sub must return undef or filehandle http://search.cpan.org/~jhi/perl-5.8.0/pod/perlfunc.pod#require 5.10+: hooks may be sub, array, or object. first level sub returns filehandle or sub or nothing http://search.cpan.org/~dapm/perl-5.10.1/pod/perlfunc.pod#require Your statement and the original code suggest the docs for 5.6 are incomplete, and should be 5.6: hook may be sub or object. first level sub returns sub or undef but not filehandle That would imply that 5.8 swapped the sub for a filehandle (or that its docs are wrong) and that 5.10 reintroduced the sub. I'm confused.

Fri Nov 15 15:59:34 2013 haarg [...] haarg.org - Correspondence added

On Fri Nov 15 15:51:10 2013, DJERIUS wrote: Show quoted text

> On Fri Nov 15 15:02:56 2013, haarg wrote:

> > On Fri Nov 15 13:51:11 2013, DJERIUS wrote:

> > > On Fri Nov 15 13:03:55 2013, haarg wrote:

> > > > On Fri Nov 15 12:36:46 2013, DJERIUS wrote:

> > > > > On Thu Nov 14 07:00:17 2013, SIMONW wrote:

> > > > > > I got a prototype working by looking for the presence of > > > > > > %main::fatpacked but that's obviously not a great longterm > > > > > > solution. > > > > > > > > > > > >

> > > > > > > > > > Ok, here are my takes on this: > > > > > > > > > > https://github.com/djerius/App- > > > > > FatPacker/commit/e7051d24e8b8cc6556d3e2fcdaf00a1348e4a791 > > > > > > > > > > https://github.com/djerius/Module- > > > > > Pluggable/commit/a92be511035f6479fe5d8ff93277060150ac8e4e > > > > > > > > > > It shoves an object into @INC rather than a subroutine ref. > > > > > > > > > > I prefer this solution as it's based upon behavior (whether the > > > > > @INC > > > > > hook provides a method) rather than by name. > > > > > > > > > > That way we can define an API and it'll be completely > > > > > independent > > > > > of > > > > > knowing who is doing what.

> > > > > > > > This will need some slight adjustment to work in perl 5.6. It > > > > should > > > > just be a matter of moving the version check inside the INC sub, > > > > returning the sub rather than the filehandle.

> > > > > > Is there some subtlety I'm missing? We already know which version > > > of > > > perl is running prior to defining the INC sub; the code returns a > > > sub > > > if $] < 5.008, the object otherwise. So < 5.008 can never see a > > > filehandle. If the object is in play, we already know we're >= > > > 5.008, > > > so it can return the filehandle. > > > > > > Or...?

> > > > The 5.6 code should also use an object. The only difference should > > be > > in what is returned from the INC sub.

> > I'm obviously not up on my Perl archeology. > > Digging through the docs for require I find this: > > 5.6: no mention of INC hooks > http://search.cpan.org/~gsar/perl-5.6.1/pod/perlfunc.pod#require > > 5.8: hooks may be sub, array, or object. first level sub must return > undef or filehandle > http://search.cpan.org/~jhi/perl-5.8.0/pod/perlfunc.pod#require > > 5.10+: hooks may be sub, array, or object. first level sub returns > filehandle or sub or nothing > http://search.cpan.org/~dapm/perl-5.10.1/pod/perlfunc.pod#require > > Your statement and the original code suggest the docs for 5.6 are > incomplete, and should be > > 5.6: hook may be sub or object. first level sub returns sub or undef > but not filehandle > > That would imply that 5.8 swapped the sub for a filehandle (or that > its docs are wrong) and that > 5.10 reintroduced the sub. > > I'm confused.

The docs are incomplete. From perl 5.6 forward, objects, subs, and arrayrefs are all supported. The only difference is that perl 5.6 doesn't have PerlIO, so you can't open a filehandle to an in memory string. It uses an alternate form for the hook return value to work around that.

Fri Nov 15 16:01:02 2013 djerius [...] cpan.org - Correspondence added

On Fri Nov 15 15:59:34 2013, haarg wrote: Show quoted text

> On Fri Nov 15 15:51:10 2013, DJERIUS wrote:

> > On Fri Nov 15 15:02:56 2013, haarg wrote:

> > > On Fri Nov 15 13:51:11 2013, DJERIUS wrote:

> > > > On Fri Nov 15 13:03:55 2013, haarg wrote:

> > > > > On Fri Nov 15 12:36:46 2013, DJERIUS wrote:

> > > > > > On Thu Nov 14 07:00:17 2013, SIMONW wrote:

> > > > > > > I got a prototype working by looking for the presence of > > > > > > > %main::fatpacked but that's obviously not a great longterm > > > > > > > solution. > > > > > > > > > > > > > >

> > > > > > > > > > > > Ok, here are my takes on this: > > > > > > > > > > > > https://github.com/djerius/App- > > > > > > FatPacker/commit/e7051d24e8b8cc6556d3e2fcdaf00a1348e4a791 > > > > > > > > > > > > https://github.com/djerius/Module- > > > > > > Pluggable/commit/a92be511035f6479fe5d8ff93277060150ac8e4e > > > > > > > > > > > > It shoves an object into @INC rather than a subroutine ref. > > > > > > > > > > > > I prefer this solution as it's based upon behavior (whether > > > > > > the > > > > > > @INC > > > > > > hook provides a method) rather than by name. > > > > > > > > > > > > That way we can define an API and it'll be completely > > > > > > independent > > > > > > of > > > > > > knowing who is doing what.

> > > > > > > > > > This will need some slight adjustment to work in perl 5.6. It > > > > > should > > > > > just be a matter of moving the version check inside the INC > > > > > sub, > > > > > returning the sub rather than the filehandle.

> > > > > > > > Is there some subtlety I'm missing? We already know which > > > > version > > > > of > > > > perl is running prior to defining the INC sub; the code returns a > > > > sub > > > > if $] < 5.008, the object otherwise. So < 5.008 can never see a > > > > filehandle. If the object is in play, we already know we're >= > > > > 5.008, > > > > so it can return the filehandle. > > > > > > > > Or...?

> > > > > > The 5.6 code should also use an object. The only difference should > > > be > > > in what is returned from the INC sub.

> > > > I'm obviously not up on my Perl archeology. > > > > Digging through the docs for require I find this: > > > > 5.6: no mention of INC hooks > > http://search.cpan.org/~gsar/perl-5.6.1/pod/perlfunc.pod#require > > > > 5.8: hooks may be sub, array, or object. first level sub must return > > undef or filehandle > > http://search.cpan.org/~jhi/perl-5.8.0/pod/perlfunc.pod#require > > > > 5.10+: hooks may be sub, array, or object. first level sub returns > > filehandle or sub or nothing > > http://search.cpan.org/~dapm/perl-5.10.1/pod/perlfunc.pod#require > > > > Your statement and the original code suggest the docs for 5.6 are > > incomplete, and should be > > > > 5.6: hook may be sub or object. first level sub returns sub or undef > > but not filehandle > > > > That would imply that 5.8 swapped the sub for a filehandle (or that > > its docs are wrong) and that > > 5.10 reintroduced the sub. > > > > I'm confused.

> > The docs are incomplete. From perl 5.6 forward, objects, subs, and > arrayrefs are all supported. The only difference is that perl 5.6 > doesn't have PerlIO, so you can't open a filehandle to an in memory > string. It uses an alternate form for the hook return value to work > around that.

Ok. I'll code to that. Thanks for the clarification.

Sat Nov 16 21:43:35 2013 djerius [...] cpan.org - Correspondence added

On Fri Nov 15 16:01:02 2013, DJERIUS wrote: Show quoted text

> On Fri Nov 15 15:59:34 2013, haarg wrote:

Show quoted text

> > The docs are incomplete. From perl 5.6 forward, objects, subs, and > > arrayrefs are all supported. The only difference is that perl 5.6 > > doesn't have PerlIO, so you can't open a filehandle to an in memory > > string. It uses an alternate form for the hook return value to work > > around that.

> > Ok. I'll code to that. Thanks for the clarification.

I've implemented that approach, added a test, and tested the code under Perls 5.6.2, 5.8.9, 5.10.1, 5.16.2 and 5.18.0 (via the FATPACKER_TESTWITH hook in pack.t). All may be found here: https://github.com/djerius/App-FatPacker

Mon Nov 18 14:46:11 2013 haarg [...] haarg.org - Correspondence added

On Sat Nov 16 21:43:35 2013, DJERIUS wrote: Show quoted text

> On Fri Nov 15 16:01:02 2013, DJERIUS wrote:

> > On Fri Nov 15 15:59:34 2013, haarg wrote:

> > > The docs are incomplete. From perl 5.6 forward, objects, subs, and > > > arrayrefs are all supported. The only difference is that perl 5.6 > > > doesn't have PerlIO, so you can't open a filehandle to an in memory > > > string. It uses an alternate form for the hook return value to > > > work > > > around that.

> > > > Ok. I'll code to that. Thanks for the clarification.

> > > I've implemented that approach, added a test, and tested the code > under Perls > > 5.6.2, 5.8.9, 5.10.1, 5.16.2 and 5.18.0 > > (via the FATPACKER_TESTWITH hook in pack.t). > > All may be found here: https://github.com/djerius/App-FatPacker

I've merged this, with a few tweaks. I'll put out a new release pretty soon.

Mon Nov 18 14:46:13 2013 haarg [...] haarg.org - Status changed from 'open' to 'patched'

Thu Nov 28 08:04:47 2013 haarg [...] haarg.org - Correspondence added

Released in 0.010000

Thu Nov 28 08:04:47 2013 haarg [...] haarg.org - Status changed from 'patched' to 'resolved'

Thu Nov 28 08:04:48 2013 haarg [...] haarg.org - Fixed in 0.010000 added