Skip Menu |

This queue is for tickets about the Term-ANSIColor CPAN distribution.

Report information
The Basics
Id: 111552
Status: resolved
Priority: 0/
Queue: Term-ANSIColor

People
Owner: RRA [...] cpan.org
Requestors: me [...] eboxr.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: 4.05



Subject: Reduce memory footprint of Term::ANSIColor with perl522
Term::ANSIColor is using 70% more memory than it was in perl514 with perl522. We can reduce this bloat to less than 10% by avoiding to import Carp and simplifying some \w regexp as we do not need unicode here. FYI, the bloat is coming from http://perl5.git.perl.org/perl.git/commitdiff/bcb875216f24899d543c036aebdba0835f8d22e6 View https://rt.perl.org/Public/Bug/Display.html?id=127392 for more details
Subject: 0001-Reduce-memory-footprint-of-Term-ANSIColor-with-perl5.patch
From 219ddda181886fa511b4f4a92ca31d7f4d98d25d Mon Sep 17 00:00:00 2001 From: Nicolas R <cpan@eboxr.com> Date: Fri, 29 Jan 2016 11:25:18 -0600 Subject: [PATCH] Reduce memory footprint of Term::ANSIColor with perl522 Term::ANSIColor is using 70% more memory than it was in perl514. We can reduce this bloat to less than 10% by avoiding to import Carp and simplifying some \w regexp. --- lib/Term/ANSIColor.pm | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/lib/Term/ANSIColor.pm b/lib/Term/ANSIColor.pm index ace4d47..1f317fe 100644 --- a/lib/Term/ANSIColor.pm +++ b/lib/Term/ANSIColor.pm @@ -23,7 +23,6 @@ use 5.006; use strict; use warnings; -use Carp qw(croak); use Exporter (); # use Exporter plus @ISA instead of use base for 5.6 compatibility. @@ -202,6 +201,11 @@ if (exists $ENV{ANSI_COLORS_ALIASES}) { } } +sub croak { + require Carp; + return Carp::croak( @_ ); +} + # Stores the current color stack maintained by PUSHCOLOR and POPCOLOR. This # is global and therefore not threadsafe. our @COLORSTACK; @@ -236,7 +240,9 @@ our @COLORSTACK; ## no critic (ClassHierarchies::ProhibitAutoloading) ## no critic (Subroutines::RequireArgUnpacking) sub AUTOLOAD { - my ($sub, $attr) = $AUTOLOAD =~ m{ \A ([\w:]*::([[:upper:]\d_]+)) \z }xms; + my ( $sub ) = $AUTOLOAD =~ /^([a-zA-Z0-9:_]+)$/; # untaint + my $attr = ( split('::', $sub ) )[-1]; + undef $attr if ( $attr && $attr !~ qr{^[A-Z0-9_]+$} ) || $sub !~ qr{::}; # Check if we were called with something that doesn't look like an # attribute. @@ -501,7 +507,7 @@ sub coloralias { return $ATTRIBUTES_R{ $ALIASES{$alias} }; } } - if ($alias !~ m{ \A [\w._-]+ \z }xms) { + if ( $alias !~ m{ \A [a-zA-Z0-9._-]+ \z }xms ) { croak(qq{Invalid alias name "$alias"}); } elsif ($ATTRIBUTES{$alias}) { croak(qq{Cannot alias standard color "$alias"}); -- 2.7.0
On 2016-01-29 12:31:57, atoomic wrote: Show quoted text
> Term::ANSIColor is using 70% more memory than it was in perl514 with > perl522. > > We can reduce this bloat to less than 10% by avoiding to import > Carp and simplifying some \w regexp as we do not need unicode here. > > FYI, the bloat is coming from > http://perl5.git.perl.org/perl.git/commitdiff/bcb875216f24899d543c036aebdba0835f8d22e6 > > View https://rt.perl.org/Public/Bug/Display.html?id=127392 for more > details
The patch does not use the same code style as the original (e.g. the original code does not have extra spaces after and before parentheses).
Subject: Re: [rt.cpan.org #111552] Reduce memory footprint of Term::ANSIColor with perl522
Date: Sun, 31 Jan 2016 00:18:27 -0800
To: "Atoomic via RT" <bug-Term-ANSIColor [...] rt.cpan.org>
From: Russ Allbery <rra [...] cpan.org>
"Atoomic via RT" <bug-Term-ANSIColor@rt.cpan.org> writes: Show quoted text
> Fri Jan 29 12:31:57 2016: Request 111552 was acted upon. > Transaction: Ticket created by atoomic > Queue: Term-ANSIColor > Subject: Reduce memory footprint of Term::ANSIColor with perl522 > Broken in: (no value) > Severity: (no value) > Owner: Nobody > Requestors: me@eboxr.com > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=111552 >
Show quoted text
> Term::ANSIColor is using 70% more memory than it was in perl514 with > perl522.
Show quoted text
> We can reduce this bloat to less than 10% by avoiding to import > Carp and simplifying some \w regexp as we do not need unicode here.
Thanks! I'll take a look. I'm unenthused by having to work around Carp being bloated. :( I feel like this is putting the program burden in the wrong place, and it's weird to have to add these strange workarounds to other modules. It would be nice if Carp were just more efficient; there are various approaches it could take internally to not load code until it's actually called. But, well, I don't feel like trying to fix Carp at the moment, so I guess I should work around it. :) Show quoted text
> sub AUTOLOAD { > - my ($sub, $attr) = $AUTOLOAD =~ m{ \A ([\w:]*::([[:upper:]\d_]+)) \z }xms; > + my ( $sub ) = $AUTOLOAD =~ /^([a-zA-Z0-9:_]+)$/; # untaint > + my $attr = ( split('::', $sub ) )[-1]; > + undef $attr if ( $attr && $attr !~ qr{^[A-Z0-9_]+$} ) || $sub !~ qr{::}; > > # Check if we were called with something that doesn't look like an > # attribute. > @@ -501,7 +507,7 @@ sub coloralias { > return $ATTRIBUTES_R{ $ALIASES{$alias} }; > } > } > - if ($alias !~ m{ \A [\w._-]+ \z }xms) { > + if ( $alias !~ m{ \A [a-zA-Z0-9._-]+ \z }xms ) { > croak(qq{Invalid alias name "$alias"}); > } elsif ($ATTRIBUTES{$alias}) { > croak(qq{Cannot alias standard color "$alias"});
The second change is fine. The first change is really awkward and strange, and makes the code much harder to read. Is all the machinery with split actually necessary, or can I just replace \w with a-zA-Z0-9_ and replace [:upper:] with A-Z and achieve the same end? I'd much prefer that, and it seems like that would get out of the locale problem that you said was causing the bloat. -- #!/usr/bin/perl -- Russ Allbery, Just Another Perl Hacker $^=q;@!>~|{>krw>yn{u<$$<[~||<Juukn{=,<S~|}<Jwx}qn{<Yn{u<Qjltn{ > 0gFzD gD, 00Fz, 0,,( 0hF 0g)F/=, 0> "L$/GEIFewe{,$/ 0C$~> "@=,m,|,(e 0.), 01,pnn,y{ rw} >;,$0=q,$,,($_=$^)=~y,$/ C-~><@=\n\r,-~$:-u/ #y,d,s,(\$.),$1,gee,print
This perfectly fine if you replace \w by A-Z0-9_ in the first AUTOLOAD regexp with the suggesting original patch (which increase difficulty to read & maintain ): VmRSS: 2572 kB with simply replacing \w => my ($sub, $attr) = $AUTOLOAD =~ m{ \A ([A-Z0-9_:]*::([[:upper:]\d_]+)) \z }xms; VmRSS: 2588 kB So I think this second option is definitely the way to go, the few extra kB worth the maintainability. Your analyze on Carp is correct, but would move the conversation to totally different one... The Carp fix I'm suggesting is simply a "load it on demand" (if we need it, which in most of the cases we dont). Thanks for considering these changes, in a future release. Let me know if you want me to provide you an updated patch. nicolas On Sun Jan 31 03:19:51 2016, RRA wrote: Show quoted text
> "Atoomic via RT" <bug-Term-ANSIColor@rt.cpan.org> writes: >
> > Fri Jan 29 12:31:57 2016: Request 111552 was acted upon. > > Transaction: Ticket created by atoomic > > Queue: Term-ANSIColor > > Subject: Reduce memory footprint of Term::ANSIColor with perl522 > > Broken in: (no value) > > Severity: (no value) > > Owner: Nobody > > Requestors: me@eboxr.com > > Status: new > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=111552 >
>
> > Term::ANSIColor is using 70% more memory than it was in perl514 with > > perl522.
>
> > We can reduce this bloat to less than 10% by avoiding to import > > Carp and simplifying some \w regexp as we do not need unicode here.
> > Thanks! I'll take a look. > > I'm unenthused by having to work around Carp being bloated. :( I > feel > like this is putting the program burden in the wrong place, and it's > weird > to have to add these strange workarounds to other modules. It would > be > nice if Carp were just more efficient; there are various approaches it > could take internally to not load code until it's actually called. > But, > well, I don't feel like trying to fix Carp at the moment, so I guess I > should work around it. :) >
> > sub AUTOLOAD { > > - my ($sub, $attr) = $AUTOLOAD =~ m{ \A > > ([\w:]*::([[:upper:]\d_]+)) \z }xms; > > + my ( $sub ) = $AUTOLOAD =~ /^([a-zA-Z0-9:_]+)$/; # untaint > > + my $attr = ( split('::', $sub ) )[-1]; > > + undef $attr if ( $attr && $attr !~ qr{^[A-Z0-9_]+$} ) || $sub !~ > > qr{::}; > > > > # Check if we were called with something that doesn't look like an > > # attribute. > > @@ -501,7 +507,7 @@ sub coloralias { > > return $ATTRIBUTES_R{ $ALIASES{$alias} }; > > } > > } > > - if ($alias !~ m{ \A [\w._-]+ \z }xms) { > > + if ( $alias !~ m{ \A [a-zA-Z0-9._-]+ \z }xms ) { > > croak(qq{Invalid alias name "$alias"}); > > } elsif ($ATTRIBUTES{$alias}) { > > croak(qq{Cannot alias standard color "$alias"});
> > The second change is fine. The first change is really awkward and > strange, and makes the code much harder to read. Is all the machinery > with split actually necessary, or can I just replace \w with a-zA-Z0- > 9_ > and replace [:upper:] with A-Z and achieve the same end? I'd much > prefer > that, and it seems like that would get out of the locale problem that > you > said was causing the bloat.