| Subject: | YAML::Dump() and YAML::Load() apparently cause memory leaks |
While trying to hunt down memory leak problems in OTRS (http://otrs.org), I stumbled over
apparent memory leaks in the YAML module.
Please find attached a small test script to reproduce this problem. On my system, this
produces the following
output:
###
Checking leaks in YAML::Dump() by performing 10000 function calls
The following symbol table differences were found:
ARRAY changed from 488 to 1924
CODE changed from 217 to 1232
GLOB changed from 468 to 1814
HASH changed from 103 to 10201
REF changed from 70 to 153
REF-ARRAY changed from 49 to 103
SCALAR changed from 3910 to 56606
Checking leaks in YAML::Load() by performing 10000 function calls
The following symbol table differences were found:
ARRAY changed from 1924 to 2030
GLOB changed from 1814 to 1951
HASH changed from 10202 to 20209
SCALAR changed from 56623 to 78348
###
This test was performed on:
YAML 0.72
This is perl, v5.10.0 built for darwin-thread-multi-2level
(with 2 registered patches, see perl -V for more detail)
Darwin otrs-mg.fritz.box 10.6.0 Darwin Kernel Version 10.6.0: Wed Nov 10 18:13:17 PST
2010; root:xnu-
1504.9.26~3/RELEASE_I386 i386
So the outcome would be that for _every_ Dump() and Load() call, YAML causes leaked
memory (additional symbol table entries). I tested this on MacOS. If I am right, this is
problematic, because long-running processes using YAML will consume constantly more
memory.
On another linux system, the result is less problematic, but still shows leaks:
###
Checking leaks in YAML::Dump() by performing 10000 function calls
The following symbol table differences were found:
ARRAY changed from 526 to 1995
CODE changed from 227 to 1256
GLOB changed from 435 to 1676
HASH changed from 125 to 261
REF changed from 79 to 166
REF-ARRAY changed from 55 to 111
REGEXP changed from 64 to 214
SCALAR changed from 4152 to 16982
Checking leaks in YAML::Load() by performing 10000 function calls
The following symbol table differences were found:
ARRAY changed from 1995 to 2108
GLOB changed from 1676 to 1751
REGEXP changed from 214 to 284
SCALAR changed from 17000 to 18727
###
This test was performed on:
YAML 0.72
This is perl 5, version 12, subversion 1 (v5.12.1) built for i586-linux-thread-multi
Linux ub-opensuse113 2.6.34.7-0.7-default #1 SMP 2010-12-13 11:13:53 +0100 i686
i686 i386 GNU/Linux
The attached test script is quite straightforward. You need to install Devel::Gladiator. Note
that tests during the installation of this module may fail, but it still works well on my test
systems.
Please let me know if I can provide further information about this problem, don't hesitate to
contact me at mg dot pub at gmx dot net.
I would be very happy to hear about this issue, if you can confirm and also possibly fix it.
With best regards,
Martin Gruner
| Subject: | yaml_leaks.pl |
#!/usr/bin/perl -w
use strict;
use warnings;
use Devel::Gladiator ();
use YAML ();
=head1 SYNOPSIS
Helper script to check for leaks in YAML.
It seems that the YAML module in its latest version (0.72) suffers from memory leaks.
This script intends to show that Dump() increases the amount of variables
in the symbol table with every call. This would mean that long-running processes grow constantly.
How is this achieved? We use the module Devel::Gladiator (on installation, some tests were failing
on my platform MacOS, but it seems to work well!) for this purpose. It takes a 'snapsnot' of the
symbol table before and after running the tests. Then these are compared and differences highlighted.
=cut
my $DumperCount = 10_000;
print "Checking leaks in YAML::Dump() by performing $DumperCount function calls\n";
# take snapshot of symbol table before tests
my $OldSymbolTable = Devel::Gladiator::arena_ref_counts();
for (1 .. $DumperCount) {
YAML::Dump('123');
}
print "The following symbol table differences were found:\n";
# take snapshot of symbol table after tests
my $NewSymbolTable = Devel::Gladiator::arena_ref_counts();
for my $Key (sort keys %{$NewSymbolTable}) {
if ($NewSymbolTable->{$Key} > (($OldSymbolTable->{$Key} || 0) + 50) ) {
warn " $Key changed from $OldSymbolTable->{$Key} to $NewSymbolTable->{$Key}\n";
}
}
my $LoadCount = 10_000;
print "Checking leaks in YAML::Load() by performing $LoadCount function calls\n";
# take snapshot of symbol table before tests
$OldSymbolTable = Devel::Gladiator::arena_ref_counts();
for (1 .. $DumperCount) {
YAML::Load("--- 123\n");
}
print "The following symbol table differences were found:\n";
# take snapshot of symbol table after tests
$NewSymbolTable = Devel::Gladiator::arena_ref_counts();
for my $Key (sort keys %{$NewSymbolTable}) {
if ($NewSymbolTable->{$Key} > (($OldSymbolTable->{$Key} || 0) + 50) ) {
warn " $Key changed from $OldSymbolTable->{$Key} to $NewSymbolTable->{$Key}\n";
}
}
# DEBUG: print out entire information
#warn Devel::Gladiator::arena_table(); # prints counts keyed by class