Skip Menu |

This queue is for tickets about the Net_SSLeay.pm CPAN distribution.

Report information
The Basics
Id: 13043
Status: resolved
Priority: 0/
Queue: Net_SSLeay.pm

People
Owner: rafl [...] debian.org
Requestors: mrdamnfrenchy [...] yahoo.com
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 1.25
Fixed in: (no value)



Subject: net::ssleay::handle don't get destroyed properly
Using POE, which uses Net::SSLeay::Handle for HTTPS request, I noticed that long running processes had way too many file descriptors open. A bit of investigation later, it turns out that Net::SSLeay::Handle creates circular references to its handles, so a handle that goes out of scope is still stored by the module, which means that unless it was explicitely closed, the socket would stay open. So, long story short, here's a patch to fix this. It also fixes the problem mentioned in bug 3920. The old version used to tie the socket itself, and use the socket's fileno to lookup the object. I modified it to tie the object directly, hence I don't need a lookup table... -Mathieu
From: mrdamnfrenchy [...] yahoo.com
Here's a test that shows whether file descriptors are retained, and when the SSL handles are destroyed. Should be after use, not at the end of the test.
# This test came about when I found out that Poco::Client::HTTP does not # release file descriptors after an HTTPS fetch. # # So to test this, I measure the numbers of open file descriptors when # starting, and then fetch a URL a number of times. The test passes if the # number of FDs is the same after all the fetches are done. As a double check, # I also measure the number of fds opened before starting the kernel, and when # its done. # # You can configure which URLs to fetch, but I just picked one http and one # https that prove the point. # # TODO: Replace the sub "count_fds" with a more platform neutral version. Right # now, this function only works on platform with the /proc file system. # Runtime config our $debuglevel = 2; # 0 none, 1 info, 2 debug my @urls = ( "www.bacus.pt", "chase.com" ); #Net::SSLeay::Handle->debug(100); # End of config use strict; use warnings; use Net::SSLeay::Handle qw/shutdown/; use Symbol qw(gensym); use Test::More; plan tests => 1+@urls; sub DEBUG(@) { $debuglevel >= 2 && print STDERR @_,"\n"; } sub INFO(@) { $debuglevel >= 1 && print STDERR @_,"\n"; } sub Net::SSLeay::Handle::DESTROY { warn "Handle $_[0] being destroyed.\n"; } sub Net::SSLeay::Handle::UNTIE { warn "Handle $_[0] is untied.\n"; } my $orig_fdcount = count_fds(); DEBUG "Orig fdcound: $orig_fdcount."; for my $url ( @urls ) { DEBUG "Getting '$url'."; { my ($host, $port) = ($url, 443); my $ssl = gensym(); tie(*$ssl, "Net::SSLeay::Handle", $host, $port); print $ssl "GET / HTTP/1.0\r\n\r\n"; shutdown($ssl,1); my @response = <$ssl>; #while (<$ssl>) { push @response, $_; } my $total_length = length(join('',@response)); DEBUG shift @response; DEBUG "Total reponse length: $total_length."; #DEBUG "Response: ".join('',@response); #close $ssl; } # Handle should have been garbage collected my $newfdcount = count_fds(); is($newfdcount, $orig_fdcount, "As many file descriptors as we started with."); } my $newfdcount = count_fds(); is($newfdcount, $orig_fdcount, "Final: As many file descriptors as we started with."); DEBUG "fdcount: $orig_fdcount/$newfdcount."; exit 0; sub count_fds { use File::Spec; my $fdpath = File::Spec->devnull(); open(F, $fdpath) or die "Can't open '$fdpath': $!\n"; my $count = fileno(F); close(F); return $count; }
From: mrdamnfrenchy [...] yahoo.com
apparently, the patch did not upload, here it is.
--- Handle.pm 2003-08-17 03:13:00.000000000 -0400 +++ ..\..\lib\Net\SSLeay\Handle.pm 2005-05-31 16:29:35.277905000 -0400 @@ -24,7 +24,6 @@ #============================================================================== my $Initialized; #-- only _initialize() once -my %Filenum_Object; #-- hash of hashes, keyed by fileno() my $Debug = 0; #-- pretty hokey my %Glob_Ref; #-- used to make unique \*S names for versions < 5.6 @@ -53,20 +52,20 @@ $Debug and print "Cipher '" . Net::SSLeay::get_cipher($ssl) . "'\n"; - $Filenum_Object{$fileno} = { + my $self = bless { ssl => $ssl, ctx => $ctx, socket => $socket, fileno => $fileno, - }; + }, $class; - return bless $socket, $class; + return $self; } sub PRINT { - my $socket = shift; + my $self = shift; - my $ssl = _get_ssl($socket); + my $ssl = _get_ssl($self); my $resp = 0; for my $msg (@_) { defined $msg or last; @@ -76,15 +75,23 @@ } sub READLINE { - my $socket = shift; - my $ssl = _get_ssl($socket); - my $line = Net::SSLeay::ssl_read_until($ssl); - return $line ? $line : undef; + my $self = shift; + my $ssl = _get_ssl($self); + if ( wantarray ) { + my @lines; + while ( my $line = Net::SSLeay::ssl_read_until($ssl) ) { + push @lines, $line; + } + return @lines; + } else { + my $line = Net::SSLeay::ssl_read_until($ssl); + return $line ? $line : undef; + } } sub READ { - my ($socket, $buf, $len, $offset) = \ (@_); - my $ssl = _get_ssl($$socket); + my ($self, $buf, $len, $offset) = \ (@_); + my $ssl = _get_ssl($self); defined($$offset) or return length($$buf = Net::SSLeay::ssl_read_all($ssl, $$len)); @@ -98,28 +105,27 @@ } sub WRITE { - my $socket = shift; + my $self = shift; my ($buf, $len, $offset) = @_; $offset = 0 unless defined $offset; # Return number of characters written. - my $ssl = $socket->_get_ssl(); + my $ssl = $self->_get_ssl(); return $len if Net::SSLeay::write($ssl, substr($buf, $offset, $len)); return undef; } sub CLOSE { - my $socket = shift; - my $fileno = fileno($socket); + my $self = shift; + my $fileno = fileno($self->{socket}); $Debug > 10 and print "close($fileno)\n"; - my $self = $socket->_get_self(); - delete $Filenum_Object{$fileno}; + Net::SSLeay::free ($self->{ssl}); Net::SSLeay::CTX_free ($self->{ctx}); - close $socket; + close $self->{socket}; } -sub FILENO { fileno($_[0]) } +sub FILENO { fileno($_[0]->{socket}) } #== Exportable Functions ===================================================== @@ -133,10 +139,10 @@ #------------------------------------------------------------------------------ sub shutdown { - my ($socket, @params) = @_; + my ($obj, @params) = @_; - my $obj = _get_self($socket); - $obj and $socket = $obj->{socket}; + #my $obj = _get_self($socket); + my $socket = UNIVERSAL::isa($obj,"Net::SSLeay::Handle") ? $obj->{socket} : $obj; return shutdown($socket, @params); } @@ -162,7 +168,7 @@ my $dest_ip = gethostbyname( $phost || $host); my $host_params = sockaddr_in($pport, $dest_ip); - my $socket = $^V ? $class->_glob_ref("$host:$port") : undef; + my $socket = $^V ? undef : $class->_glob_ref("$host:$port"); socket($socket, &PF_INET(), &SOCK_STREAM(), 0) or die "socket: $!"; connect($socket, $host_params) or die "connect: $!"; @@ -222,7 +228,7 @@ #------------------------------------------------------------------------------ sub _get_self { - return $Filenum_Object{fileno(shift)}; + return shift; } #--- _get_ssl($socket) -------------------------------------------------------- @@ -232,8 +238,7 @@ #------------------------------------------------------------------------------ sub _get_ssl { - my $socket = shift; - return $Filenum_Object{fileno($socket)}->{ssl}; + return $_[0]->{ssl}; } 1;
On Tue May 31 16:44:26 2005, guest wrote: Show quoted text
> Using POE, which uses Net::SSLeay::Handle for HTTPS request, I noticed > that long running processes had way too many file descriptors open. > > A bit of investigation later, it turns out that Net::SSLeay::Handle > creates circular references to its handles, so a handle that goes > out of scope is still stored by the module, which means that unless > it was explicitely closed, the socket would stay open. > > So, long story short, here's a patch to fix this. It also fixes the > problem mentioned in bug 3920. > > The old version used to tie the socket itself, and use the socket's > fileno to lookup the object. I modified it to tie the object > directly, hence I don't need a lookup table... > > -Mathieu
Thanks for your report. I fixed the problem in SVN and it will be part of the upcoming release. I pretty much applied your patch with the exception of some minor things and the fact that I didn't modify the behaviour of READLINE. I generally think it's a good idea, but I fear breaking backwards compatibility. Are you aware of any piece of software that uses Net::SSLeay::Handle and relies on the fact that READLINE only returns a single line, even in list context? Regards, Flo
Subject: Re: [rt.cpan.org #13043] net::ssleay::handle don't get destroyed properly
Date: Sat, 21 Jan 2006 13:54:07 -0800 (PST)
To: bug-Net_SSLeay.pm [...] rt.cpan.org
From: Mathieu Longtin <mrdamnfrenchy [...] yahoo.com>
No. I use it has part of POE::Component::HTTP::Client and LWP::Get. If those were working before, lets leave them as that. OTOH, I've been using the fixed version of Net::SSLeay::Handle since I posted that bug, and those two modules have been working fine. -Mathieu --- Florian Ragwitz via RT <bug-Net_SSLeay.pm@rt.cpan.org> wrote: Show quoted text
> Thanks for your report. I fixed the problem in SVN and it > will be part of the upcoming release. > > I pretty much applied your patch with the exception of > some minor things and the fact that I didn't modify the > behaviour of READLINE. I generally think it's a good
idea, Show quoted text
> but I fear breaking backwards compatibility. Are you > aware of any piece of software that uses > Net::SSLeay::Handle and relies on the fact that READLINE > only returns a single line, even in list context? > > Regards, > Flo
Show quoted text
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
CC: net-ssleay-devel [...] lists.alioth.debian.org
Subject: Re: [rt.cpan.org #13043] net::ssleay::handle don't get destroyed properly
Date: Sun, 22 Jan 2006 01:29:50 +0100
To: "mrdamnfrenchy [...] yahoo.com via RT" <bug-Net_SSLeay.pm [...] rt.cpan.org>
From: Florian Ragwitz <rafl [...] debian.org>
On Sat, Jan 21, 2006 at 04:54:54PM -0500, mrdamnfrenchy@yahoo.com via RT wrote: Show quoted text
> No. I use it has part of POE::Component::HTTP::Client and > LWP::Get. If those were working before, lets leave them as > that. > > OTOH, I've been using the fixed version of > Net::SSLeay::Handle since I posted that bug, and those two > modules have been working fine. >
OK, I think I'll change the behaviour then and leave a note in the README or the changelog. Your patch will be part of the next release. Thank you very much. -Flo Show quoted text
> --- Florian Ragwitz via RT <bug-Net_SSLeay.pm@rt.cpan.org> > wrote: >
> > Thanks for your report. I fixed the problem in SVN and it > > will be part of the upcoming release. > > > > I pretty much applied your patch with the exception of > > some minor things and the fact that I didn't modify the > > behaviour of READLINE. I generally think it's a good
> idea,
> > but I fear breaking backwards compatibility. Are you > > aware of any piece of software that uses > > Net::SSLeay::Handle and relies on the fact that READLINE > > only returns a single line, even in list context?
Download signature.asc
application/pgp-signature 189b

Message body not shown because it is not plain text.

RT-Send-CC: net-ssleay-devel [...] lists.alioth.debian.org
Your changes were slightly modified, applied to svn trunk and will be part of the next release. Therefor I'm closing this bug now. Thanks, Flo