Skip Menu |

This queue is for tickets about the Net-RabbitMQ CPAN distribution.

Report information
The Basics
Id: 60075
Status: resolved
Priority: 0/
Queue: Net-RabbitMQ

People
Owner: Nobody in particular
Requestors: BUNDACIA [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Critical
Broken in: 0.1.6
Fixed in: (no value)



Subject: Memory Leak in net_rabbitmq_recv()
The net_rabbitmq_recv suffers from the memory leak bug described here: http://perldoc.perl.org/perlxs.html#Returning-SVs%2c-AVs-and-HVs-through-RETVAL The result is that if you have a long running process that pulls jobs from a Rabbit queue it will eventually run out of memory. I've included a diff that will fix the leak. Even with this change there still seems to be a slow leak, but it's an order of magnitude less. --- current/Net-RabbitMQ-0.1.6/RabbitMQ.xs 2010-07-18 13:04:33.000000000 -0400 +++ new/Net-RabbitMQ-0.1.6/RabbitMQ.xs 2010-08-04 13:09:34.463954263 -0400 @@ -384,6 +384,7 @@ int result = 0; CODE: RETVAL = newHV(); + sv_2mortal((SV*)RETVAL); result = internal_recv(RETVAL, conn, 0); if(result <= 0) Perl_croak(aTHX_ "Bad frame read."); OUTPUT:
I found the source of the other leak. Here's a complete diff with includes the first fix as well as the fix for the other leaks: tlittle@mysql1 ~/rabbit $ diff -u current/Net-RabbitMQ-0.1.6/RabbitMQ.xs new/Net- RabbitMQ-0.1.6/RabbitMQ.xs --- current/Net-RabbitMQ-0.1.6/RabbitMQ.xs 2010-07-18 13:04:33.000000000 -0400 +++ new/Net-RabbitMQ-0.1.6/RabbitMQ.xs 2010-08-04 13:46:31.696055051 -0400 @@ -92,7 +92,7 @@ HV *props; props = newHV(); - hv_store(RETVAL, "props", strlen("props"), newRV((SV *)props), 0); + hv_store(RETVAL, "props", strlen("props"), newRV_noinc((SV *)props), 0); p = (amqp_basic_properties_t *) frame.payload.properties.decoded; if (p->_flags & AMQP_BASIC_CONTENT_TYPE_FLAG) { @@ -384,6 +384,7 @@ int result = 0; CODE: RETVAL = newHV(); + sv_2mortal((SV*)RETVAL); result = internal_recv(RETVAL, conn, 0); if(result <= 0) Perl_croak(aTHX_ "Bad frame read."); OUTPUT: @@ -533,7 +534,7 @@ rv = internal_recv(hv, conn, 1); if(rv <= 0) Perl_croak(aTHX_ "Bad frame read."); } - RETVAL = (SV *)newRV((SV *)hv); + RETVAL = (SV *)newRV_noinc((SV *)hv); } else RETVAL = &PL_sv_undef; On Wed Aug 04 13:17:11 2010, BUNDACIA wrote: Show quoted text
> The net_rabbitmq_recv suffers from the memory leak bug described here: > http://perldoc.perl.org/perlxs.html#Returning-SVs%2c-AVs-and-HVs- > through-RETVAL > > The result is that if you have a long running process that pulls jobs > from a Rabbit queue it > will eventually run out of memory. I've included a diff that will fix > the leak. Even with this > change there still seems to be a slow leak, but it's an order of > magnitude less. > > --- current/Net-RabbitMQ-0.1.6/RabbitMQ.xs 2010-07-18 > 13:04:33.000000000 -0400 > +++ new/Net-RabbitMQ-0.1.6/RabbitMQ.xs 2010-08-04 13:09:34.463954263 > -0400 > @@ -384,6 +384,7 @@ > int result = 0; > CODE: > RETVAL = newHV(); > + sv_2mortal((SV*)RETVAL); > result = internal_recv(RETVAL, conn, 0); > if(result <= 0) Perl_croak(aTHX_ "Bad frame read."); > OUTPUT:
CC: Theo Schlossnagle <jesus [...] omniti.com>
Subject: Re: [rt.cpan.org #60075] Memory Leak in net_rabbitmq_recv()
Date: Wed, 4 Aug 2010 14:35:17 -0400
To: bug-Net-RabbitMQ [...] rt.cpan.org
From: Theo Schlossnagle <jesus [...] omniti.com>
Applied. On Aug 4, 2010, at 2:05 PM, Trevor Little via RT wrote: Show quoted text
> Queue: Net-RabbitMQ > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=60075 > > > I found the source of the other leak. Here's a complete diff with includes the first fix as well > as the fix for the other leaks: > > tlittle@mysql1 ~/rabbit $ diff -u current/Net-RabbitMQ-0.1.6/RabbitMQ.xs new/Net- > RabbitMQ-0.1.6/RabbitMQ.xs > --- current/Net-RabbitMQ-0.1.6/RabbitMQ.xs 2010-07-18 13:04:33.000000000 -0400 > +++ new/Net-RabbitMQ-0.1.6/RabbitMQ.xs 2010-08-04 13:46:31.696055051 -0400 > @@ -92,7 +92,7 @@ > > HV *props; > props = newHV(); > - hv_store(RETVAL, "props", strlen("props"), newRV((SV *)props), 0); > + hv_store(RETVAL, "props", strlen("props"), newRV_noinc((SV *)props), 0); > > p = (amqp_basic_properties_t *) frame.payload.properties.decoded; > if (p->_flags & AMQP_BASIC_CONTENT_TYPE_FLAG) { > @@ -384,6 +384,7 @@ > int result = 0; > CODE: > RETVAL = newHV(); > + sv_2mortal((SV*)RETVAL); > result = internal_recv(RETVAL, conn, 0); > if(result <= 0) Perl_croak(aTHX_ "Bad frame read."); > OUTPUT: > @@ -533,7 +534,7 @@ > rv = internal_recv(hv, conn, 1); > if(rv <= 0) Perl_croak(aTHX_ "Bad frame read."); > } > - RETVAL = (SV *)newRV((SV *)hv); > + RETVAL = (SV *)newRV_noinc((SV *)hv); > } > else > RETVAL = &PL_sv_undef; > > On Wed Aug 04 13:17:11 2010, BUNDACIA wrote:
>> The net_rabbitmq_recv suffers from the memory leak bug described here: >> http://perldoc.perl.org/perlxs.html#Returning-SVs%2c-AVs-and-HVs- >> through-RETVAL >> >> The result is that if you have a long running process that pulls jobs >> from a Rabbit queue it >> will eventually run out of memory. I've included a diff that will fix >> the leak. Even with this >> change there still seems to be a slow leak, but it's an order of >> magnitude less. >> >> --- current/Net-RabbitMQ-0.1.6/RabbitMQ.xs 2010-07-18 >> 13:04:33.000000000 -0400 >> +++ new/Net-RabbitMQ-0.1.6/RabbitMQ.xs 2010-08-04 13:09:34.463954263 >> -0400 >> @@ -384,6 +384,7 @@ >> int result = 0; >> CODE: >> RETVAL = newHV(); >> + sv_2mortal((SV*)RETVAL); >> result = internal_recv(RETVAL, conn, 0); >> if(result <= 0) Perl_croak(aTHX_ "Bad frame read."); >> OUTPUT:
> > >
-- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle
Subject: Re: [rt.cpan.org #60075] Memory Leak in net_rabbitmq_recv()
Date: Wed, 4 Aug 2010 15:03:46 -0400
To: bug-Net-RabbitMQ [...] rt.cpan.org
From: Trevor Little <trevormg19 [...] gmail.com>
Great, thanks! Do you think you'll be pushing a new version of the module to the cpan in the near future? If so, I'll just wait to update to the official new version instead of patching ours on our production systems. -bundacia On Wed, Aug 4, 2010 at 2:35 PM, Theo Schlossnagle via RT <bug-Net-RabbitMQ@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=60075 > > > Applied. > > On Aug 4, 2010, at 2:05 PM, Trevor Little via RT wrote: >
>>       Queue: Net-RabbitMQ >> Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=60075 > >> >> I found the source of the other leak. Here's a complete diff with includes the first fix as well >> as the fix for the other leaks: >> >> tlittle@mysql1 ~/rabbit $ diff -u current/Net-RabbitMQ-0.1.6/RabbitMQ.xs new/Net- >> RabbitMQ-0.1.6/RabbitMQ.xs >> --- current/Net-RabbitMQ-0.1.6/RabbitMQ.xs      2010-07-18 13:04:33.000000000 -0400 >> +++ new/Net-RabbitMQ-0.1.6/RabbitMQ.xs  2010-08-04 13:46:31.696055051 -0400 >> @@ -92,7 +92,7 @@ >> >>     HV *props; >>     props = newHV(); >> -    hv_store(RETVAL, "props", strlen("props"), newRV((SV *)props), 0); >> +    hv_store(RETVAL, "props", strlen("props"), newRV_noinc((SV *)props), 0); >> >>     p = (amqp_basic_properties_t *) frame.payload.properties.decoded; >>     if (p->_flags & AMQP_BASIC_CONTENT_TYPE_FLAG) { >> @@ -384,6 +384,7 @@ >>     int result = 0; >>   CODE: >>     RETVAL = newHV(); >> +    sv_2mortal((SV*)RETVAL); >>     result = internal_recv(RETVAL, conn, 0); >>     if(result <= 0) Perl_croak(aTHX_ "Bad frame read."); >>   OUTPUT: >> @@ -533,7 +534,7 @@ >>         rv = internal_recv(hv, conn, 1); >>         if(rv <= 0) Perl_croak(aTHX_ "Bad frame read."); >>       } >> -      RETVAL = (SV *)newRV((SV *)hv); >> +      RETVAL = (SV *)newRV_noinc((SV *)hv); >>     } >>     else >>       RETVAL = &PL_sv_undef; >> >> On Wed Aug 04 13:17:11 2010, BUNDACIA wrote:
>>> The net_rabbitmq_recv suffers from the memory leak bug described here: >>> http://perldoc.perl.org/perlxs.html#Returning-SVs%2c-AVs-and-HVs- >>> through-RETVAL >>> >>> The result is that if you have a long running process that pulls jobs >>> from a Rabbit queue it >>> will eventually run out of memory. I've included a diff that will fix >>> the leak. Even with this >>> change there still seems to be a slow leak, but it's an order of >>> magnitude less. >>> >>> --- current/Net-RabbitMQ-0.1.6/RabbitMQ.xs      2010-07-18 >>> 13:04:33.000000000 -0400 >>> +++ new/Net-RabbitMQ-0.1.6/RabbitMQ.xs  2010-08-04 13:09:34.463954263 >>> -0400 >>> @@ -384,6 +384,7 @@ >>>     int result = 0; >>>   CODE: >>>     RETVAL = newHV(); >>> +    sv_2mortal((SV*)RETVAL); >>>     result = internal_recv(RETVAL, conn, 0); >>>     if(result <= 0) Perl_croak(aTHX_ "Bad frame read."); >>>   OUTPUT:
>> >> >>
> > -- > Theo Schlossnagle > http://omniti.com/is/theo-schlossnagle > > > > > > >
fixed in v0.1.7