| Subject: | Apparent memory leaks in connect() and prepare() |
While trying to hunt down memory leak problems in OTRS (http://otrs.org), I stumbled over apparent memory
leaks in the DBI module.
Please find attached a small test script to reproduce this problem. On my system, this produces the following
output:
###
Checking leaks in DBI->connect() by performing 10000 calls to connect() and disconnect()
The following symbol table differences were found:
ARRAY changed from 636 to 10727
CODE changed from 465 to 525
GLOB changed from 779 to 1033
HASH changed from 306 to 438
REF changed from 306 to 463
REF-HASH changed from 184 to 327
SCALAR changed from 5911 to 8496
Checking leaks in DBI->prepare() by performing 10000 calls to prepare()
The following symbol table differences were found:
ARRAY changed from 10731 to 20732
###
This test was performed on:
DBI 1.616
This is perl, v5.10.0 built for darwin-thread-multi-2level
(with 2 registered patches, see perl -V for more detail)
Darwin otrs-mg.fritz.box 10.6.0 Darwin Kernel Version 10.6.0: Wed Nov 10 18:13:17 PST 2010; root:xnu-
1504.9.26~3/RELEASE_I386 i386
So the outcome would be that for _every_ connect() and prepare() call, DBI causes leaked memory (additional
symbol table entries). I tested this on MacOS with both DBD::mysql and DBD::Pg.
If I am right, this is problematic, because long-running processes using DBI will consume constantly more
memory.
On another linux system, the result is less problematic, but still shows leaks in connect():
###
Checking leaks in DBI->connect() by performing 10000 calls to connect() and disconnect()
The following symbol table differences were found:
ARRAY changed from 696 to 792
CODE changed from 479 to 539
GLOB changed from 754 to 990
HASH changed from 286 to 426
REF changed from 259 to 416
REF-HASH changed from 133 to 276
SCALAR changed from 6415 to 9023
Checking leaks in DBI->prepare() by performing 10000 calls to prepare()
The following symbol table differences were found:
###
This test was performed on:
DBI 1.609 (similar results with 1.616 on this system)
This is perl 5, version 12, subversion 1 (v5.12.1) built for i586-linux-thread-multi
Linux ub-opensuse113 2.6.34.7-0.7-default #1 SMP 2010-12-13 11:13:53 +0100 i686 i686 i386 GNU/Linux
The attached test script is quite straightforward. You need a mysql database to connect to, change the
connection settings as appropriate, and install Devel::Gladiator. Note that tests during the installation of this
module may fail, but it still works well on my test systems.
Please let me know if I can provide further information about this problem, don't hesitate to contact me at mg
dot pub at gmx dot net.
I would be very happy to hear about this issue, if you can confirm and also possibly fix it.
With best regards,
Martin Gruner
| Subject: | dbi_leaks.pl |
#!/usr/bin/perl -w
use strict;
use warnings;
use Devel::Gladiator ();
use DBI ();
=head1 SYNOPSIS
Helper script to check for leaks in DBI.
It seems that the DBI module in its latest version (1.616) suffers from memory leaks.
This script intends to show that connect() and prepare() increase the amount of variables
in the symbol table with every call. This would mean that long-running processes grow constantly.
How is this achieved? We use the module Devel::Gladiator (on installation, some tests were failing
on my platform MacOS, but it seems to work well!) for this purpose. It takes a 'snapsnot' of the
symbol table before and after running the tests. Then these are compared and differences highlighted.
=cut
# You may have to adapt this to an existing database on your system.
# It can be empty, without tables.
my $DSN = 'DBI:mysql:database=otrs31_dev;host=localhost;';
my $DB = 'otrs31_dev';
my $PW = 'otrs31_dev';
my $ConnectCount = 10_000;
print "Checking leaks in DBI->connect() by performing $ConnectCount calls to connect() and disconnect()\n";
# take snapshot of symbol table before tests
my $OldSymbolTable = Devel::Gladiator::arena_ref_counts();
for (1 .. $ConnectCount) {
my $DBH = DBI->connect(
$DSN,
$DB,
$PW,
{
mysql_auto_reconnect => 0,
},
);
$DBH->disconnect();
}
print "The following symbol table differences were found:\n";
# take snapshot of symbol table after tests
my $NewSymbolTable = Devel::Gladiator::arena_ref_counts();
for my $Key (sort keys %{$NewSymbolTable}) {
if ($NewSymbolTable->{$Key} > (($OldSymbolTable->{$Key} || 0) + 50) ) {
warn " $Key changed from $OldSymbolTable->{$Key} to $NewSymbolTable->{$Key}\n";
}
}
my $PrepareCount = 10_000;
print "\n\nChecking leaks in DBI->prepare() by performing $PrepareCount calls to prepare()\n";
my $DBH = DBI->connect(
$DSN,
$DB,
$PW,
{
mysql_auto_reconnect => 0,
},
);
# take snapshot of symbol table before tests
$OldSymbolTable = Devel::Gladiator::arena_ref_counts();
for (1 .. $PrepareCount) {
my $STH = $DBH->prepare('SELECT 1 LIMIT 1');
# optional: these calls do not change the leak result
$STH->execute();
$STH->fetchall_arrayref();
$STH->finish();
}
$DBH->disconnect();
print "The following symbol table differences were found:\n";
# take snapshot of symbol table after tests
$NewSymbolTable = Devel::Gladiator::arena_ref_counts();
for my $Key (sort keys %{$NewSymbolTable}) {
if ($NewSymbolTable->{$Key} > (($OldSymbolTable->{$Key} || 0) + 50) ) {
warn " $Key changed from $OldSymbolTable->{$Key} to $NewSymbolTable->{$Key}\n";
}
}
# DEBUG: print out entire information
#warn Devel::Gladiator::arena_table(); # prints counts keyed by class