Bug #44493 for threads: memory eating 1.72

Mon Mar 23 17:47:55 2009 christian [...] hbr1.com - Ticket created

Subject:	memory eating 1.72
Date:	Mon, 23 Mar 2009 06:26:18 +0100 (CET)
To:	bug-threads [...] rt.cpan.org
From:	"Christian Mueller" <christian [...] hbr1.com>

when i tested a new module for threadsafety i found a bug in the perl threading model. the following test doesn't stop eating memory. i tested it with perl/5.10.0 and perl/5.8.8 and threads 1.72 on linux x64 use threads; retry: $t = threads->create( \&thread_sub ); $t->join; goto retry; sub thread_sub { }

Mon Mar 23 21:02:05 2009 post-christian [...] freenet.de - Correspondence added

ps. it does not happen on windows (active perl/5.10.0, threads/1.7.1). On Mo. 23. Mär. 2009, 17:47:55, christian@hbr1.com wrote: Show quoted text

> when i tested a new module for threadsafety i found a bug in the perl > threading model. the following test doesn't stop eating memory. i

tested Show quoted text

> it with perl/5.10.0 and perl/5.8.8 and threads 1.72 on linux x64 > > use threads; > > retry: > > $t = threads->create( \&thread_sub ); > $t->join; > > goto retry; > > sub thread_sub { > } > >

Mon Mar 23 21:02:05 2009 The RT System itself - Status changed from 'new' to 'open'

Wed Mar 25 09:41:28 2009 jdhedden [...] cpan.org - Correspondence added

CC:	christian [...] hbr1.com
Subject:	Re: [rt.cpan.org #44493] memory eating 1.72
Date:	Wed, 25 Mar 2009 09:41:17 -0400
To:	pp <perl5-porters [...] perl.org>, David Mitchell <davem [...] iabyn.com>, bug-threads [...] rt.cpan.org
From:	"Jerry D. Hedden" <jdhedden [...] cpan.org>

Christian Mueller wrote: Show quoted text

> Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=44493 > > > when i tested a new module for threadsafety i found a bug in the perl > threading model. the following test doesn't stop eating memory. i tested > it with perl/5.10.0 and perl/5.8.8 and threads 1.72 on linux x64 > > use threads; > > retry: > > $t = threads->create( \&thread_sub ); > $t->join; > > goto retry; > > sub thread_sub { > }

He also wrote: Show quoted text

> it does not happen on windows (active perl/5.10.0, threads/1.7.1).

Hi, Dave, et al, Is this really a bug or is it a matter of lagging to clean up memory from a fast loop that creates and destroys threads?

Wed Mar 25 10:56:21 2009 davem [...] iabyn.com - Correspondence added

CC:	pp <perl5-porters [...] perl.org>, bug-threads [...] rt.cpan.org, christian [...] hbr1.com
Subject:	Re: [rt.cpan.org #44493] memory eating 1.72
Date:	Wed, 25 Mar 2009 14:55:51 +0000
To:	"Jerry D. Hedden" <jdhedden [...] cpan.org>
From:	Dave Mitchell <davem [...] iabyn.com>

On Wed, Mar 25, 2009 at 09:41:17AM -0400, Jerry D. Hedden wrote: Show quoted text

> Christian Mueller wrote:

> > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=44493 > > > > > when i tested a new module for threadsafety i found a bug in the perl > > threading model. the following test doesn't stop eating memory. i tested > > it with perl/5.10.0 and perl/5.8.8 and threads 1.72 on linux x64 > > > > use threads; > > > > retry: > > > > $t = threads->create( \&thread_sub ); > > $t->join; > > > > goto retry; > > > > sub thread_sub { > > }

> > He also wrote:

> > it does not happen on windows (active perl/5.10.0, threads/1.7.1).

> > Hi, Dave, et al, > Is this really a bug or is it a matter of lagging to clean up > memory from a fast loop that creates and destroys threads?

On the second and subsequent thread creations, $t holds a pointer to the remains of the previous thread; this gets cloned into the new thread, and so the previous thread never gets freed. Rinse and repeat. So its not a bug, just a non-obvious consequence of the fact that a new thread inherits a copy of everything in the old thread. If you undef $t before each new thread creation, the problem goes away: ... undef $t; $t = threads->create( \&thread_sub ); -- The Enterprise successfully ferries an alien VIP from one place to another without serious incident. -- Things That Never Happen in "Star Trek" #7

Wed Mar 25 10:59:01 2009 jdhedden [...] cpan.org - Status changed from 'open' to 'rejected'

Wed Mar 25 13:10:49 2009 post-christian [...] freenet.de - Correspondence added

Show quoted text

> On the second and subsequent thread creations, $t holds a pointer to > the remains of the previous thread; this gets cloned into the new

thread, Show quoted text

> and so the previous thread never gets freed. Rinse and repeat. > > So its not a bug, just a non-obvious consequence of the fact that a

new Show quoted text

> thread inherits a copy of everything in the old thread. If you undef

$t Show quoted text

> before each new thread creation, the problem goes away: > > ... > undef $t; > $t = threads->create( \&thread_sub ); > > >

use threads; retry: undef $t; $t = threads->create( \&thread_sub ); $t->join; undef $t; goto retry; sub thread_sub { undef $t; } sorry, but it makes no different. it still eats up the memory.

Wed Mar 25 13:10:50 2009 The RT System itself - Status changed from 'rejected' to 'open'

Wed Mar 25 13:17:02 2009 post-christian [...] freenet.de - Correspondence added

without any global variable, and still eating memory... use threads; use Time::HiRes; retry: threads->create( \&thread_sub ); Time::HiRes::usleep( 10_000 ); goto retry; sub thread_sub { #print threads->self->tid, "\n"; threads->self->detach; }

Tue Jan 19 17:54:15 2010 jdhedden [...] cpan.org - Correspondence added

I have updated the threads POD to note that this situation exists, and added the advice to reuse threads in long-lived apps. I've also created an example script that illustrates this concept. These will go out with the next release (whenever that occurs). On Wed Mar 25 13:17:02 2009, CHRMUE wrote: Show quoted text

> without any global variable, and still eating memory... > > use threads; > use Time::HiRes; > > retry: > > threads->create( \&thread_sub ); > > Time::HiRes::usleep( 10_000 ); > goto retry; > > sub thread_sub { > #print threads->self->tid, "\n"; > threads->self->detach; > }

Wed Apr 14 16:35:54 2010 jdhedden [...] cpan.org - Status changed from 'open' to 'resolved'

Wed Apr 14 16:35:55 2010 jdhedden [...] cpan.org - Fixed in 1.77 added