Bug #7805 for Module-Pluggable: Module::Pluggable has a large overhead

Tue Sep 28 22:13:29 2004 adamk [...] cpan.org - Ticket created

Subject:

Module::Pluggable has a large overhead

One of the main goals of the Perl Email Project, working in the Email:: namepsace, is to be small, tight and modular to as great a degree as possible. After encountering some dependency bugs, the first complete dependency mapping of Email:: has been completed, to help identify some of the areas which are causing Email:: to bloat out unnescesarily. See this URL before we continue http://pep.kwiki.org/index.cgi?ProjectBloat Looking at the area at the right, you can see the problem. While Module::Pluggable was great for doing Email::Abstract (and potentially converting over Email::Send as well) it is introducing quite a bit of bloat. Namely 5 additional non-core packages and 1200k of memory load overhead. Most of the memory bloat comes from File::Find::Rule, which I notice you don't use very heavily, just a very simple name('*.pm') query. The other one, UNIVERSAL::require seems to just provide some syntactic sugar. To help make it easier to spread the use of Module::Pluggable, and to help with out bloat problem, it would be great if we could cut away these external dependencies. Which would mean replacing the use of the 1100k File::Find::Rule with a small precompiled 20-50k &wanted function, and replacing UNIVERSAL::require with normal eval { require ... } stuff. I know it's a little uglier, but I think it would be great to have Module::Pluggable with no non-core dependencies at all. It would make it much easier to use and distribute/bundle. If you are amenable, I'd be happy to write the initial patch for it. For and questions, talk to myself (Alias) or cwest on irc.perl.org #perl.

Fri Oct 01 06:23:45 2004 simon [...] thegestalt.org - Correspondence added

Date:	Fri, 1 Oct 2004 11:17:02 +0100
From:	Simon Wistow <simon [...] thegestalt.org>
To:	via RT <bug-Module-Pluggable [...] rt.cpan.org>
Subject:	Re: [cpan #7805] Module::Pluggable has a large overhead
RT-Send-Cc:

Show quoted text

> Which would mean replacing the use of the 1100k File::Find::Rule with > a small precompiled 20-50k &wanted function, and replacing > UNIVERSAL::require with normal eval { require ... } stuff.

I've been meaning to do this for a while - I've just not had the necessary kick int he ass to do it. Show quoted text

> I know it's a little uglier, but I think it would be great to have > Module::Pluggable with no non-core dependencies at all. It would make > it much easier to use and distribute/bundle.

Agreed. Show quoted text

> If you are amenable, I'd be happy to write the initial patch for it. > For and questions, talk to myself (Alias) or cwest on irc.perl.org > #perl.

I'm away on holiday at the moment (and committing the faux pas of checking email when on holiday :( ) so if you wnat to send a patch that'd be cool - otherwise I'll get on it when I get back (early next week) Simon -- i try not to let pedantry get in the way of a glib remark

Sat Oct 02 14:10:16 2004 adamk [...] cpan.org - Correspondence added

From:

cpan [...] ali.as

Done This is, of course, just the module file itself, and not the Makefile as well

? Pluggable.pm.patch Index: Pluggable.pm =================================================================== RCS file: /var/lib/cvs/CPAN/modules/Module/Pluggable.pm,v retrieving revision 1.1 retrieving revision 1.2 diff -r1.1 -r1.2 5c5,6 < use File::Find::Rule qw/find/; --- > use Cwd (); > use File::Find (); 8d8 < use UNIVERSAL::require; 17c17 < $VERSION = '2.0'; --- > $VERSION = '2.1'; 33c33 < --- > 41,42d40 < < 229c227 < next unless ( -e $sp && -d $sp ); --- > next unless ( -e $sp && -d _ ); # Use the cached stat the second time 233c231,240 < my @files = find( name => "*.pm", in => [$sp] ); --- > my $cwd = Cwd::getcwd; > my @files = (); > @files = File::Find::find( > sub { # Inlined from File::Find::Rule C< name => '*.pm' > > return unless (m((?-xism:^(?=[^\\.])[^/]*\\.pm$)) ); > (my $path = $File::Find::name) =~ s#^\\./##; > push @files, $path; > }, > $sp ); > chdir $cwd; 253c260,261 < $_->require or carp "Couldn't require $_ : $UNIVERSAL::require::ERROR"; --- > eval "CORE::require $_"; > carp "Couldn't require $_ : $@" if $@; 312d319 <

Sat Oct 02 14:13:05 2004 adamk [...] cpan.org - Correspondence added

I did a quite load test with those changes, and while we don't save all of the overhead, due to still needing File::Find and Cwd, we still recover 400-500k. I'm not entirely sure if we need Cwd or not, but that's what File::Find::Rule did internally, and what's there is exactly what F:F:R did for your use of it, with the useless statements chopped out of course. I also added a small change to use a cached stat in one place too.

Thu Oct 07 10:26:16 2004 simonw [...] cpan.org - Correspondence added

Integrated with some CVS stuff and released as 2.2 (skipping 2.1 dev version) I had to frig the 'wanted' subroutine slightly.

Thu Oct 07 10:26:22 2004 simonw [...] cpan.org - Given to SIMONW

Thu Oct 07 10:26:22 2004 simonw [...] cpan.org - Status changed from 'new' to 'resolved'