| CC: | ether [...] cpan.org, HMBRAND [...] cpan.org |
| Subject: | patch for SSL bug |
Many bug-reports for Net::HTTP and LWP are probably resolved by this proposed patch. (This is
a duplicate from https://rt.cpan.org/Ticket/Display.html?id=104115 )
Until Net::HTTP 6.03, the simplified socket read logic looked like this: (Net::HTTP::Methods::my_readline() and my_read()
my $buf = '';
while(1) {
my $n = $self->sysread($buf, 1024, length $buf);
defined $n || $!{EAGAIN} or last;
select(undef, undef, undef, 0.1);
}
Since Net::HTTP 6.04, the sysread is event-driven, not polling anymore. More like this
my $buf = '';
my $rbits; vec($rbits, fileno($sock), 1) = 1;
while(1) {
select($rbits, undef, undef, $timeout);
my $n = $self->sysread($buf, 1024, length $buf);
defined $n || $!{EAGAIN} or last;
}
The file-handle (usually a socket) where sysread() gets its data from may have more than 1024
bytes available! This happens often when there is a SSL layer between the program and the
socket.
In "the new" code, we sleep on select(). An 'r' event in select() flags whether new data has
become available since the previous call to select(). It does *not* show whether there still
is data available to be read!!! So, when there is more than 1024 bytes coming in at once, the
remaining bytes get stuck! The select used by my_read()/can_read will wait for new data before
it will trigger a new 'r' although there is still data available.
The patch for Net::HTTP::Method v6.07 attached
| Subject: | sysread.patch |
--- Methods.pm 2014-07-24 07:35:58.000000000 +0200
+++ Methods.pm.new 2015-04-30 10:15:15.627895209 +0200
@@ -265,20 +265,34 @@
if $max_line_length && length($_) > $max_line_length;
# need to read more data to find a line ending
+ my $new_bytes = 0;
+
READ:
- {
- die "read timeout" unless $self->can_read;
- my $n = $self->sysread($_, 1024, length);
- unless (defined $n) {
- redo READ if $!{EINTR} || $!{EAGAIN};
- # if we have already accumulated some data let's at least
- # return that as a line
- die "$what read failed: $!" unless length;
- }
- unless ($n) {
- return undef unless length;
- return substr($_, 0, length, "");
+ { # wait until bytes start arriving
+ $self->can_read
+ or die "read timeout";
+
+ # consume all incoming bytes
+ while(1) {
+ my $bytes_read = $self->sysread($_, 1024, length);
+ if(defined $bytes_read) {
+ $new_bytes += $bytes_read;
+ last if $bytes_read < 1024;
+ }
+ elsif($!{EINTR} || $!{EAGAIN} || $!{EWOULDBLOCK}) {
+ redo READ;
+ }
+ else {
+ # if we have already accumulated some data let's at
+ # least return that as a line
+ length or die "$what read failed: $!";
+ last;
+ }
}
+
+ # no line-ending, no new bytes
+ return length($_) ? substr($_, 0, length($_), "") : undef
+ if $new_bytes==0;
}
}
die "$what line too long ($pos; limit is $max_line_length)"