| CC: | bug-Compress-LZ4 [...] rt.cpan.org |
| Subject: | Memory leak in LZ4 |
| Date: | Thu, 31 May 2012 20:10:50 -0700 (PDT) |
| From: | Eric Wheeler <perl-bugs [...] o.ewheeler.net> |
Hello Gray,
It looks like +8MB of virtual memory is allocated (but never used) when
Compress::LZ4 attempts to decompress an invalid stream, thus leading to a
huge VM overcommit. Here's a short exmple:
use Compress::LZ4;
foreach (1..100)
{
system("ps -p $$ -o vsz"); # VM commit size in kB
$a = decompress('xxx');
}
system("ps -p $$ -o vsz"); # VM commit size in kB
Most systems will kill a process when it attempts to overcommit memory
beyond a certain point. For now I've set /proc/sys/vm/overcommit_limit=1
as a bandaid.
Is there a way to have decompress check the magic bits before attempting
to decompress? I didn't see an obvious "islz4()"-like function in LZ4.xs,
but please point me in the right direction if such a thing exists. Right
now the only way I can test if a stream is LZ4 is to decompress and check
for undef.
Thanks for your help!
-Eric
--
Eric Wheeler
888-LINUX26 (888-546-8926)
www.GlobalLinuxSecurity.pro
President
eWheeler, Inc.
dba Global Linux Security
PO Box 14707
Portland, OR 97286