Skip Menu |

This queue is for tickets about the Module-Runtime CPAN distribution.

Report information
The Basics
Id: 86394
Status: resolved
Priority: 0/
Queue: Module-Runtime

People
Owner: Nobody in particular
Requestors: DOY [...] cpan.org
ether [...] cpan.org
haarg [...] haarg.org
Cc: NNUTTER [...] cpan.org
ribasushi [...] leporine.io
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: use_package_optimistically fails to report a failure in the module itself to load a different module
If you use use_package_optimistically to attempt to load a module with an error, that error is typically reported, except if the error is due to a failure in that module to load another module. This is because the regex used in use_package_optimistically is too lax. See attached for a fix.
Subject: upo.patch
diff -uNr a/lib/Module/Runtime.pm b/lib/Module/Runtime.pm --- a/lib/Module/Runtime.pm 2012-02-16 15:11:34.000000000 -0500 +++ b/lib/Module/Runtime.pm 2013-06-24 18:24:05.251462576 -0400 @@ -381,8 +381,9 @@ my($name, $version) = @_; check_module_name($name); eval { local $SIG{__DIE__}; require_module($name); }; + my $file = module_notional_filename($name); die $@ if $@ ne "" && - $@ !~ /\ACan't locate .+ at \Q@{[__FILE__]}\E line/s; + $@ !~ /\ACan't locate \Q$file\E .+ at \Q@{[__FILE__]}\E line/s; $name->VERSION($version) if defined $version; return $name; } diff -uNr a/t/upo.t b/t/upo.t --- a/t/upo.t 2012-02-16 15:11:34.000000000 -0500 +++ b/t/upo.t 2013-06-24 18:23:21.651491626 -0400 @@ -1,7 +1,7 @@ use warnings; use strict; -use Test::More tests => 30; +use Test::More tests => 31; BEGIN { use_ok "Module::Runtime", qw(use_package_optimistically); } @@ -87,4 +87,8 @@ ok defined($t::Context::VERSION); ok $INC{"t/Context.pm"}; +# loads a module which doesn't exist +test_use_package_optimistically("t::WithDep"); +like $err, qr/Can't locate Dep\.pm in \@INC/; + 1; diff -uNr a/t/WithDep.pm b/t/WithDep.pm --- a/t/WithDep.pm 1969-12-31 19:00:00.000000000 -0500 +++ b/t/WithDep.pm 2013-06-24 18:22:15.438202381 -0400 @@ -0,0 +1 @@ +use Dep;
Any chance of getting this fixed and released soon?
On 2013-10-23 07:58:30, haarg wrote: Show quoted text
> Any chance of getting this fixed and released soon?
This bug is exposed by the new Moose TRIAL release. CALL THE NATIONAL GUARD! :)
Subject: Re: [rt.cpan.org #86394] use_package_optimistically fails to report a failure in the module itself to load a different module
Date: Tue, 3 Dec 2013 20:16:38 +0000
To: Jesse Luehrs via RT <bug-Module-Runtime [...] rt.cpan.org>
From: Zefram <zefram [...] fysh.org>
Jesse Luehrs via RT wrote: Show quoted text
>If you use use_package_optimistically to attempt to load a module with an >error, that error is typically reported, except if the error is due to a >failure in that module to load another module.
That certainly is a problem with use_package_optimistically(). However, this function is (and always was) documented to mostly match base.pm, and base.pm exhibits the same problem for the same reason. So while both modules would be improved by making the module-not-found detection tighter, I'm dubious about changing Module::Runtime first. I see that base.pm's RT queue does not contain any mention of this problem, so I can't even make the change in anticipation of base.pm imminently adopting the same change. I'm not absolutely ruling out making a change independently of base.pm, but I'd need some convincing to make them diverge. As for the specific patch, I'm not entirely happy that it solves the problem. It'll certainly catch most cases that the current logic gets wrong, and I believe the notional filename is used for the "Can't locate" message regardless of platform, so that much is OK. But if it's going to be made more specific then I'd rather it be made as specific as possible. The regexp could be made further specific about which line (of M/R.pm) the error is signalled on. However, it's still a hack; the failure to find the module is not being distinctively signalled by require(). I wonder about using a magic object in @INC to detect require()'s iteration, but worry about this being visible to other code. Overall, the missing-module detection is a hack that can't possibly be entirely reliable. This shouldn't be a dire problem, as it's in a function that's labelled "optimistic", and the failure mode is that it's a bit more optimistic than you'd like. What are you doing that relies on such a fragile mechanism? -zefram
For what it's worth we were about to submit this same bug report. Our use case is that we are loading configuration modules using a hierarchy so modules do not have to exist but if they do we want to know if they fail to load. I think at the very least the documents should clarify the behavior as it currently says, "If the module cannot be found then it is assumed that the package was actually already loaded by other means, and no error is signalled." and should maybe say, "If the module cannot be loaded", or, "If the module cannot be found, or fails to compile,". Keeping consistency with base.pm, while kind of saddening, does make sense. I don't know if it would be sufficient to check $@ ourselves as I am concerned about checking $@ when I can't see the eval beside it. By the way, the check we use currently is: if ($@ and $@ !~ m/^Can't locate $filename in \@INC/) { Carp::croak($@); } So clearly ours needs improvement. Also base.pm's is: die if $@ && $@ !~ /^Can't locate .*? at \(eval /; which is also different.
I don't really think trying to keep 100% compatibility with base.pm is a good idea. base.pm has the incredible weight of backwards compatibility to deal with, which isn't the case for Module::Runtime. I've tried to make improvements to base.pm in the past, but was basically told by p5p that "heuristics can't be perfect, so we won't improve them". And given that the intention of the code clearly doesn't match the implementation, I don't think being help back by base.pm is good. base.pm also has the extra (although buggy) check of the symbol table to reduce the odds of silent failures. As for what was hitting this issue, Moose is attempting to move towards more reliable checks for module loading. However, backwards compatibility concerns means it still has to make allowances for inline packages. In the course of developing the latest dev releases, this issue was masking a bug in the new Moose code, hampering the debugging process.
Key reason I care about such a thing: If I have an optional dependency of some sort, I tend to want to (a) not care if said module isn't installed at all (b) throw an exception if it's present but fails to load, because that probably indicates a problem in need of fixing. At least during development, having code automatically fall back to "eh, I don't have it" tends to hide compilation failures, and during production it tends to hide things like library compatibility issues, and I think it's reasonable to assume that if the .pm file is there at all the user intended it to work and it not doing so should die(). This is, of course, not the same as the base.pm use case and I'd be perfectly happy with a different function for that. I also suspect you're right that base.pm should be changed as well though, and would encourage somebody more awake than I am right now to file something over there.
Subject: Re: [rt.cpan.org #86394] use_package_optimistically fails to report a failure in the module itself to load a different module
Date: Tue, 3 Dec 2013 22:12:00 +0000
To: Graham Knop via RT <bug-Module-Runtime [...] rt.cpan.org>
From: Zefram <zefram [...] fysh.org>
Graham Knop via RT wrote: Show quoted text
>As for what was hitting this issue, Moose is attempting to move towards >more reliable checks for module loading. However, backwards compatibility >concerns means it still has to make allowances for inline packages.
I expect Moose could use some more reliable heuristic to detect inline packages. If this is specifically about Moose classes, surely there's a reliable way to detect whether a package has had the Moose magic applied, then you can use the non-optimistic use_module() on anything that's not already a Moose class. Even if the non-existence of a module were reliably detectable, the optimistic heuristics suck. This is not a good thing to be building into any modern framework. -zefram
CC: ether [...] cpan.org, haarg [...] haarg.org, ribasushi [...] cpan.org
Subject: Re: [rt.cpan.org #86394] use_package_optimistically fails to report a failure in the module itself to load a different module
Date: Tue, 3 Dec 2013 17:15:26 -0500
To: Zefram via RT <bug-Module-Runtime [...] rt.cpan.org>
From: Jesse Luehrs <DOY [...] cpan.org>
On Tue, Dec 03, 2013 at 05:12:11PM -0500, Zefram via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=86394 > > > Graham Knop via RT wrote:
> >As for what was hitting this issue, Moose is attempting to move towards > >more reliable checks for module loading. However, backwards compatibility > >concerns means it still has to make allowances for inline packages.
> > I expect Moose could use some more reliable heuristic to detect inline > packages. If this is specifically about Moose classes, surely there's a > reliable way to detect whether a package has had the Moose magic applied, > then you can use the non-optimistic use_module() on anything that's not > already a Moose class. > > Even if the non-existence of a module were reliably detectable, the > optimistic heuristics suck. This is not a good thing to be building > into any modern framework.
Yes, I made Moose stop using these heuristics entirely once I ran into this bug - it's no longer an issue there. That said, it does make debugging things quite a bit more confusing, as mst pointed out. -doy
Subject: Re: [rt.cpan.org #86394] use_package_optimistically fails to report a failure in the module itself to load a different module
Date: Tue, 3 Dec 2013 22:18:14 +0000
To: MSTROUT via RT <bug-Module-Runtime [...] rt.cpan.org>
From: Zefram <zefram [...] fysh.org>
MSTROUT via RT wrote: Show quoted text
>This is, of course, not the same as the base.pm use case and I'd be >perfectly happy with a different function for that.
I'd also be happier with providing a separate function that attempts a cleaner "optional module" semantic. Detecting non-existent module would still be a hack, but if it's separate from the base.pm-alike function then I'd be willing to make it hackier to get closer to the target semantic. Would this be usable for the Moose use case? It wouldn't detect inline modules, but I think Moose shouldn't be applying heuristics for that. And if it does want heuristics, it should use its own. -zefram
On Tue Dec 03 17:12:11 2013, zefram@fysh.org wrote: Show quoted text
> Graham Knop via RT wrote:
> >As for what was hitting this issue, Moose is attempting to move towards > >more reliable checks for module loading. However, backwards compatibility > >concerns means it still has to make allowances for inline packages.
> > I expect Moose could use some more reliable heuristic to detect inline > packages. If this is specifically about Moose classes, surely there's a > reliable way to detect whether a package has had the Moose magic applied, > then you can use the non-optimistic use_module() on anything that's not > already a Moose class. > > Even if the non-existence of a module were reliably detectable, the > optimistic heuristics suck. This is not a good thing to be building > into any modern framework. > > -zefram
Inline packages declared elsewhere can be assumed to be in a consistent state, because they would have already thrown errors otherwise. So checking for Moose's magic on the classes is reliable. But a half-compiled module can be in an inconsistent state. Moose's magic may be applied, even though the module failed to load. So the compile failure needs to be detected before checking for the magic. These needs exactly match the behavior described in the use_package_optimistically docs (aside from the "matches base.pm" bit). As a separate note, base.pm is blead upstream. Unfortunately that isn't reflected in the dist's metadata.
So I just went to look at fixing base.pm, and realized that it doesn't suffer from this bug (mostly). The check it uses looks like this: die if $@ && $@ !~ /^Can't locate .*? at \(eval /; The key difference is that it lacks a /s flag on the regex. So it will only ignore missing modules directly in an eval. I'm still going to file an issue about making base.pm more strict, but I think that's shows that this definitely needs to change in some way. I'd say that the /s flag should be removed in addition to this patch.
Subject: Re: [rt.cpan.org #86394] use_package_optimistically fails to report a failure in the module itself to load a different module
Date: Wed, 4 Dec 2013 11:03:36 +0000
To: Graham Knop via RT <bug-Module-Runtime [...] rt.cpan.org>
From: Zefram <zefram [...] fysh.org>
Graham Knop via RT wrote: Show quoted text
>So I just went to look at fixing base.pm, and realized that it doesn't >suffer from this bug (mostly).
It'll show up in a different set of places, that's a good point. It does fundamentally have the problem that a fairly broad set of "Can't locate" errors will be interpreted as the particular module of interest being missing. -zefram
Filed a ticket for making base.pm's check more strict: https://rt.perl.org/Ticket/Display.html?id=120685
Subject: Re: [rt.cpan.org #86394] use_package_optimistically fails to report a failure in the module itself to load a different module
Date: Wed, 4 Dec 2013 13:56:06 +0000
To: Graham Knop via RT <bug-Module-Runtime [...] rt.cpan.org>
From: Zefram <zefram [...] fysh.org>
Graham Knop via RT wrote: Show quoted text
>I'd say that the /s flag should be removed in addition to this patch.
Not so simple. See [perl #120687]. Parsing error messages that have had filenames interpolated into them is *hard*. -zefram
Here is an updated patch based on perl rt#120685.
Subject: 0001-only-ignore-direct-module-missing-errors-in-use_pack.patch
From f71ba1707c4cfb979004ea81d1f069ba47442db5 Mon Sep 17 00:00:00 2001 From: Graham Knop <haarg@haarg.org> Date: Tue, 17 Dec 2013 16:33:13 -0500 Subject: [PATCH] only ignore direct module missing errors in use_package_optimistically --- lib/Module/Runtime.pm | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/lib/Module/Runtime.pm b/lib/Module/Runtime.pm index 5217a8c..93a6c97 100644 --- a/lib/Module/Runtime.pm +++ b/lib/Module/Runtime.pm @@ -317,6 +317,14 @@ sub require_module($) { return scalar(require(&module_notional_filename)); } } +BEGIN { + my $line = __LINE__ - 4; + $line -= 4 + if _WORK_AROUND_BROKEN_MODULE_STATE; + my $location = __FILE__ . " line $line"; + *_REQUIRE_LOCATION = sub () { $location }; +} + =back @@ -379,10 +387,11 @@ function work just like L</use_module>. sub use_package_optimistically($;$) { my($name, $version) = @_; - check_module_name($name); + my $file = module_notional_filename($name); eval { local $SIG{__DIE__}; require_module($name); }; - die $@ if $@ ne "" && - $@ !~ /\ACan't locate .+ at \Q@{[__FILE__]}\E line/s; + die $@ if $@ ne "" + && $@ !~ /^Can't locate \Q$file\E .*? at \Q@{[_REQUIRE_LOCATION]}\E\.\n\z/s + || $@ =~ /Compilation failed in require at \Q@{[_REQUIRE_LOCATION]}\E\.\n\z/; $name->VERSION($version) if defined $version; return $name; } -- 1.8.5
Another patch update per updated base.pm in blead.
Subject: 0001-only-ignore-direct-module-missing-errors-in-use_pack.patch
From b9b9cd2d4b57b228c238bd7c444c56d4a92d40a3 Mon Sep 17 00:00:00 2001 From: Graham Knop <haarg@haarg.org> Date: Sat, 18 Jan 2014 22:36:23 -0500 Subject: [PATCH] only ignore direct module missing errors in use_package_optimistically --- lib/Module/Runtime.pm | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/lib/Module/Runtime.pm b/lib/Module/Runtime.pm index 5217a8c..7919de5 100644 --- a/lib/Module/Runtime.pm +++ b/lib/Module/Runtime.pm @@ -317,6 +317,16 @@ sub require_module($) { return scalar(require(&module_notional_filename)); } } +my $require_error_location_rx; +{ + my $line = __LINE__ - 5; + $line -= 4 + if _WORK_AROUND_BROKEN_MODULE_STATE; + my $location = ' at ' . __FILE__ . " line $line"; + $require_error_location_rx = + qr/\Q$location\E(?:, <[^>]*> (?:line|chunk) [0-9]+)?\.\n/; +} + =back @@ -379,10 +389,11 @@ function work just like L</use_module>. sub use_package_optimistically($;$) { my($name, $version) = @_; - check_module_name($name); + my $file = module_notional_filename($name); eval { local $SIG{__DIE__}; require_module($name); }; - die $@ if $@ ne "" && - $@ !~ /\ACan't locate .+ at \Q@{[__FILE__]}\E line/s; + die $@ if $@ ne "" + && $@ !~ /^Can't locate \Q$file\E .*?$require_error_location_rx\z/s + || $@ =~ /Compilation failed in require$require_error_location_rx\z/; $name->VERSION($version) if defined $version; return $name; } -- 1.8.5.2
On 2014-01-18 19:37:58, haarg wrote: Show quoted text
> Another patch update per updated base.pm in blead.
Is this patch acceptable? it would be really good to get this out, as we've had another ticket reported against Moose due to this.
Subject: Re: [rt.cpan.org #86394] use_package_optimistically fails to report a failure in the module itself to load a different module
Date: Thu, 6 Feb 2014 10:32:29 +0000
To: Karen Etheridge via RT <bug-Module-Runtime [...] rt.cpan.org>
From: Zefram <zefram [...] fysh.org>
Karen Etheridge via RT wrote: Show quoted text
>Is this patch acceptable? it would be really good to get this out, >as we've had another ticket reported against Moose due to this.
Will look this evening. I think I can resolve it now. -zefram
Fixed in Module-Runtime-0.014, now on CPAN.