Skip Menu |

This queue is for tickets about the List-MoreUtils CPAN distribution.

Report information
The Basics
Id: 15638
Status: resolved
Priority: 0/
Queue: List-MoreUtils

People
Owner: Nobody in particular
Requestors: tlowery [...] cc3.com
Cc:
AdminCc:

Bug Information
Severity: Critical
Broken in: 0.10
Fixed in: (no value)



Subject: Memory Leak in repeated usage of all any none
There appears to be a memory leak using the List::MoreUtils XS functions all, any, and none. Test on perl 5.6.1 and 5.8.7 Test script: use strict; use warnings; use List::MoreUtils qw{all any none}; my @checkme = qw{HI FL AK}; my @states = qw{AZ CA NV MI DE PA}; my $misses = 0; while ( 1 == 1 ) { foreach my $to_check (@checkme) { # if (all { $to_check eq $_ } @states) { # $misses++; # } # if (any { $to_check eq $_ } @states) { # $misses++; # } if (none { $to_check eq $_ } @states) { $misses++; } } } The kernel kill the process afer a minute or so. dmesg shows "Out of Memory". The problem doesn't happen when using the pure perl version (LIST_MOREUTILS_PP=1). $ perl -V Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration: Platform: osname=linux, osvers=2.4.18-4smp, archname=i686-linux uname='linux rip 2.4.18-4smp #1 smp thu may 2 18:32:34 edt 2002 i686 unknown ' config_args='-Dprefix=/u01/app/perl' hint=previous, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef Compiler: cc='cc', ccflags ='-fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' ccversion='', gccversion='2.96 20000731 (Red Hat Linux 7.3 2.96-112)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, usemymalloc=n, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lndbm -lgdbm -ldl -lm -lc -lcrypt -lutil perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil libc=/lib/libc-2.2.5.so, so=so, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib -L/u01/app/perl/lib' Characteristics of this binary (from libperl): Compile-time options: USE_LARGE_FILES Built under linux Compiled at Aug 7 2002 18:40:37 %ENV: PERL5LIB="/lf/21st/devel/lib:/lf/utils/devel/lib" @INC: /lf/21st/devel/lib /lf/utils/devel/lib/i686-linux /lf/utils/devel/lib /u01/app/perl/lib/perl5/5.6.1/i686-linux /u01/app/perl/lib/perl5/5.6.1 /u01/app/perl/lib/perl5/site_perl/5.6.1/i686-linux /u01/app/perl/rip/lib/perl5/site_perl/5.6.1 /u01/app/perl/rip/lib/perl5/site_perl .
Date: Wed, 09 Nov 2005 08:54:23 +0100
From: Tassilo von Parseval <tassilo.von.parseval [...] rwth-aachen.de>
Subject: Re: [cpan #15638] Memory Leak in repeated usage of all any none
To: Guest via RT <bug-List-MoreUtils [...] rt.cpan.org>
RT-Send-Cc:
On Tue, Nov 08, 2005 at 08:24:45PM -0500 Guest via RT wrote: Show quoted text
> > This message about List-MoreUtils was sent to you by guest <> via rt.cpan.org > > Full context and any attached attachments can be found at: > <URL: https://rt.cpan.org/Ticket/Display.html?id=15638 > > > There appears to be a memory leak using the List::MoreUtils XS functions all, any, and none. > > Test on perl 5.6.1 and 5.8.7 > > Test script: > use strict; > use warnings; > > use List::MoreUtils qw{all any none}; > > my @checkme = qw{HI FL AK}; > my @states = qw{AZ CA NV MI DE PA}; > > my $misses = 0; > > while ( 1 == 1 ) { > > foreach my $to_check (@checkme) { > # if (all { $to_check eq $_ } @states) { > # $misses++; > # } > # if (any { $to_check eq $_ } @states) { > # $misses++; > # } > if (none { $to_check eq $_ } @states) { > $misses++; > } > } > > } > > The kernel kill the process afer a minute or so. dmesg shows "Out of Memory".
Gee, that is indeed one nice memory leak. My suspicion is that this is a memory leak in perl. List::Util::first() shows the same behaviour and my module does it in the same way for its iterative functions. Skimming through the source I can find no explanation for the leak and valgrind isn't a lot more helpful. I'll raise that issue on the perl-porters list to see whether the problem is with perl or List::Util. In the latter case I just have to wait for Graham Barr to come up with a fix and incorporate it into my module. :-) Show quoted text
> The problem doesn't happen when using the pure perl version (LIST_MOREUTILS_PP=1).
Cheers, Tassilo -- use bigint; $n=71423350343770280161397026330337371139054411854220053437565440; $m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200);
[guest - Tue Nov 8 20:24:45 2005]: Show quoted text
> There appears to be a memory leak using the List::MoreUtils XS > functions all, any, and none.
And those are now fixed in 0.13. I think the leak was actually in every function taking a CODE-block as first argument. Thanks again for spotting! Tassilo