| Subject: | Memory leak in iterators with_objects which are not explicitly ->finish()ed |
In the ->get_objects method, where $return_iterator and $with_objects
are true, the code to generate an iterator does something like...
$iterator->_next_code(sub {
....
$iterator->finish;
....
$iterator->finish;
}
The calls seem to be made in order to deal with the $manual_limit case.
Whether or not the $manual_limit case is active, however, this
subroutine closes over the $iterator variable, and a reference to the
code is then stored inside $iterator itself. This is a circular
reference, and prevents the DESTROY method from being called on the
iterator until global destruction.
If one's code is relying on this automatic destruction, its failure will
induce memory leaks. This has been a problem for some of our
application's long-running processes which periodically instatiate these
iterators to run database queries.
As a workaround, $iterator->finish can be called explicitly (thereby
setting the _next_code to sub { 0 } and breaking the circular
reference). Or the handy dandy attached patch can be applied, which
makes use of the fact that a $self variable is available which refers to
the same object, but does not close over the original reference.
We experienced this problem in Rose::DB::Object::Manager where $VERSION
= '0.765'; casual visual inspection seems to indicate that it remains in
0.781.
We do not have a test case which runs independently of our (rather
large) codebase. Sorry.
| Subject: | Manager-v0.765.diff |
Index: lib/Rose/DB/Object/Manager.pm
===================================================================
--- lib/Rose/DB/Object/Manager.pm (revision 326)
+++ lib/Rose/DB/Object/Manager.pm (working copy)
@@ -2175,7 +2175,7 @@
no warnings;
if($manual_limit && $self->{'_count'} == $manual_limit)
{
- $iterator->finish;
+ $self->finish;
last ROW;
}
}
@@ -2237,7 +2237,7 @@
if($manual_limit && $self->{'_count'} == $manual_limit)
{
$self->total($self->{'_count'});
- $iterator->finish;
+ $self->finish;
}
#$Debug && warn "Return $object_class $objects[-1]{'id'}\n";