Skip Menu |

This queue is for tickets about the Template-Toolkit CPAN distribution.

Report information
The Basics
Id: 46058
Status: resolved
Worked: 3.5 hours (210 min)
Priority: 0/
Queue: Template-Toolkit

People
Owner: Nobody in particular
Requestors: garu [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 2.20
Fixed in: (no value)



Subject: memory leak
If I run this: --------------8<-------------- use strict; use warnings; use Template; my $text = '[% x.y %]'; my $out; my $tt = Template->new(); while (1) { $out = undef; $tt->process(\$text, { x => {} }, \$out); } --------------8<-------------- and go to a different terminal to watch the process with 'top' or any equivalent tool, I see memory consumption rising abruptly for the process. If I change the $tt->process() line above to either: $tt->process(\$text, {}, \$out); or $tt->process(\$text, { x => { y => 42 } }, \$out); , the leak goes away. Thoughts? Anyway, ***thank you*** for such an amazing template system :)
Hi! We were able to narrow the scope of the problem a little. Apparently it's on Template::Stash::XS -----------8<------------ # this leaks big time use Template::Stash::XS; my $stash = Template::Stash::XS->new( { x => {} } ); while (1) { $s->get( ['x', 0, 'y', 0] ); } -----------8<------------ Using 'top' or something equivalent, you can easily see it eating up a Gig of memory in just a few seconds! If I change the code above replacing Template::Stash::XS with just Template::Stash, the leak goes away. Now, I tried following the leak, but I don't have much experience debugging XS, so here's my top result: -----------8<------------ use Devel::Leak; use Template::Stash::XS; my $stash = Template::Stash::XS->new( { x => {} } ); my $handle; Devel::Leak::NoteSV($handle); $stash->get( ['x', 0] ); Devel::Leak::CheckSV($handle); ----------->8------------ This yields no output, meaning nothing leaked. However, if I change the get() statement to: $stash->get( ['x', 0, 'y', 0] ); I get the output: new 0x8a6bc80 : new 0x8928538 : Hope this helps! And thanks again for the time and attention to work on such a helpful piece of code :) Cheers, garu On Wed May 13 20:57:09 2009, GARU wrote: Show quoted text
> If I run this: > > --------------8<-------------- > use strict; > use warnings; > > use Template; > > my $text = '[% x.y %]'; > my $out; > my $tt = Template->new(); > > while (1) { > $out = undef; > $tt->process(\$text, { x => {} }, \$out); > } > --------------8<-------------- > > and go to a different terminal to watch the process with 'top' or any > equivalent tool, I see memory consumption rising abruptly for the process. > > If I change the $tt->process() line above to either: > > $tt->process(\$text, {}, \$out); > > or > > $tt->process(\$text, { x => { y => 42 } }, \$out); > > , the leak goes away. > > Thoughts? > > Anyway, ***thank you*** for such an amazing template system :)
Hi again, As another note, it appears the bug was introduced in 2.20, as we could not reproduce it in 2.19. Hope this helps narrow it down :) Cheers garu On Thu May 14 13:44:18 2009, GARU wrote: Show quoted text
> Hi! > > We were able to narrow the scope of the problem a little. Apparently > it's on Template::Stash::XS > > -----------8<------------ > # this leaks big time > use Template::Stash::XS; > > my $stash = Template::Stash::XS->new( { x => {} } ); > > while (1) { > $s->get( ['x', 0, 'y', 0] ); > } > -----------8<------------ > > Using 'top' or something equivalent, you can easily see it eating up a > Gig of memory in just a few seconds! > > If I change the code above replacing Template::Stash::XS with just > Template::Stash, the leak goes away. > > Now, I tried following the leak, but I don't have much experience > debugging XS, so here's my top result: > > -----------8<------------ > use Devel::Leak; > use Template::Stash::XS; > > my $stash = Template::Stash::XS->new( { x => {} } ); > my $handle; > > Devel::Leak::NoteSV($handle); > $stash->get( ['x', 0] ); > Devel::Leak::CheckSV($handle); > ----------->8------------ > > This yields no output, meaning nothing leaked. However, if I change the > get() statement to: > > $stash->get( ['x', 0, 'y', 0] ); > > I get the output: > > new 0x8a6bc80 : > new 0x8928538 : > > > Hope this helps! And thanks again for the time and attention to work on > such a helpful piece of code :) > > Cheers, > > garu > > On Wed May 13 20:57:09 2009, GARU wrote:
> > If I run this: > > > > --------------8<-------------- > > use strict; > > use warnings; > > > > use Template; > > > > my $text = '[% x.y %]'; > > my $out; > > my $tt = Template->new(); > > > > while (1) { > > $out = undef; > > $tt->process(\$text, { x => {} }, \$out); > > } > > --------------8<-------------- > > > > and go to a different terminal to watch the process with 'top' or any > > equivalent tool, I see memory consumption rising abruptly for the
process. Show quoted text
> > > > If I change the $tt->process() line above to either: > > > > $tt->process(\$text, {}, \$out); > > > > or > > > > $tt->process(\$text, { x => { y => 42 } }, \$out); > > > > , the leak goes away. > > > > Thoughts? > > > > Anyway, ***thank you*** for such an amazing template system :)
>
Thanks for the info and test. Once you had narrowed it down to a problem in 2.19->2.20 it was easy to find the offending code that introduced it. I had missed out an SvREFCNT_dec(root) so there was an SV whose refcnt never dropped to zero, and hence never had its memory freed. It's now fixed and I've run your test to confirm that it no longer leaks memory. The change is checked into subversion: http://template-toolkit.org/svnweb/Template2/revision/?rev=1202 I'll push out another developer release some time next week and then version v2.21 a week or so after that. Many thanks. Andy
Hmmm... I think I spoke too soon. That "fix" seems to be causing problems elsewhere. I've re-opened the bug while I investigate further...
That's ok, thanks a lot for all the trouble - hope to see a fix soon! On Fri May 15 12:09:44 2009, ABW wrote: Show quoted text
> Hmmm... I think I spoke too soon. That "fix" seems to be causing > problems elsewhere. I've re-opened the bug while I investigate further...
OK, I'm fairly confident that it really is fixed now. I've added a t/stash-xs-leak.t test script based on your original example. It leaks memory without the fix, but doesn't with it applied. http://template-toolkit.org/svnweb/Template2/revision/?rev=1203 I've released a new developer version 2.20_2 containing the fix. Cheers A