| Subject: | Use of Tie::Cache::LRU (as documented) leaks memory |
I've done some extensive testing on this problem, and determined that when Tie::Cache::LRU is used for the ->{CachedKids} field of a database handle, the database handle, all cached statement handles, and the CachedKids itself do not get freed when the database handle goes out of scope.
Tie::Cache::LRU does not cause this problem on its own, even when used in a similar structure to DBI.
DBI does not exhibit the problem unless it is used with Tie::Cache::LRU.
Some interaction between DBI and Tie::Cache::LRU causes it to fail to garbage-collect. I'm reporting this as a bug in DBI because the documentation officially recommends Tie::Cache::LRU to solve the problem of limiting the number prepared statements.
The following are some easy ways to re-produce my findings:
# This is a minimal example that does leak, and is prescribed by the
# DBI documentation. It leaks on all of:
# Linux, perl 5.20.0, DBI 1.633, MySQL backend
# Linux, perl 5.12.3, DBI 1.634, MySQL backend
# Linux, perl 5.8.9, DBI 1.634, MySQL backend
# -> and warns about freeing unreferenced scalars during global destruction!
# Win32, perl 5.20.2, DBI 1.633, ODBC SQLServer backend
perl -MScalar::Util -MTie::Cache::LRU -MDevel::Gladiator -MDBI -e '
use strict; use warnings;
my ($dsn, $u, $p, $table)= @ARGV;
for (1..100) {
my $dbh= DBI->connect($dsn, $u, $p, {}) // last;
{
my $cache;
my $t= tie %$cache, "Tie::Cache::LRU", 10;
$dbh->{CachedKids}= $cache;
}
$dbh->prepare_cached("select count(*) from $table");
}
print Devel::Gladiator::arena_table()' DSN USER PASS TABLE | grep Tie
# This parody of DBI data structures does not leak perl objects
# when created/destroyed in a loop.
perl -MScalar::Util -MTie::Cache::LRU -MDevel::Gladiator -e '
for (1..100) {
my %container;
tie %container, "Tie::Cache::LRU", 100;
my $dbh= \%container;
my %cache;
tie %cache, "Tie::Cache::LRU", 100;
$dbh->{CachedKids}= \%cache;
$dbh->{CachedKids}{1}= bless {}, "main";
$dbh->{CachedKids}{2} ||= bless {}, "main";
Scalar::Util::weaken( $dbh->{CachedKids}{2}{Database}= $dbh );
}'
These are excerpts from the DBI.pm module:
----------------------------------------------------------------------------
Because of the way the Perl tie mechanism works you cannot reliably
use the C<||=> operator directly to initialise the attribute, like this:
my $foo = $dbh->{private_yourmodname_foo} ||= { ... }; # WRONG
you should use a two step approach like this:
my $foo = $dbh->{private_yourmodname_foo};
$foo ||= $dbh->{private_yourmodname_foo} = { ... };
----------------------------------------------------------------------------
my $cache = $dbh->{CachedKids} ||= {};
----------------------------------------------------------------------------