| Subject: | Net::HTTP chunk handling broken for non-blocking sockets |
Hi,
the chunk handling of Net::HTTP(::Methods) is broken for non-blocking sockets.
This affects LWP with IO::Socket::SSL as https backend.
Net::SSL as https backend is not affected because it explicitly disables non-blocking support, and http is not affected either, because it overrides
sysread to be blocking even if the socket is non-blocking.
Details:
in Net::HTTP::Methods::read_entity_body we have
472 if (defined $chunked) {
473 # The state encoded in $chunked is:
474 # $chunked == 0: read CRLF after chunk, then chunk header
475 # $chunked == -1: read chunk header
476 # $chunked > 0: bytes left in current chunk to read
477
478 if ($chunked <= 0) {
479 my $line = my_readline($self, 'Entity body');
...
513 my $n = $chunked;
514 $n = $size if $size && $size < $n;
515 $n = my_read($self, $$buf_ref, $n);
516 return undef unless defined $n;
517
518 ${*$self}{'http_chunked'} = $chunked - $n;
if $chunked was <=0 in line 478 it will try to read the end of the last chunk (only if chunked==0) followed by the reading of the next chunk header
to determine the size. It will then set $chunked to the size of the next chunk.
If everything was successful, it will try to read the next chunk in line 515.
For blocking sockets this will block until some data are read (or fatal error occurs) and then it will continue in line 518 and update the
http_chunked attribute of $self with the number of bytes still to read.
But for non-blocking sockets the read might fail. In this case the method will return in line 516 without updating the http_chunked attribute, so
that it will later retry to read the chunk header starting in line 478.
Thus the fix should be:
$n = my_read($self, $$buf_ref, $n);
+ ${*$self}{'http_chunked'} = $chunked - $n||0;
return undef unless defined $n;
- ${*$self}{'http_chunked'} = $chunked - $n;
Doing only this patch would result in an busy loop calling read_entity_body until data are available.
Thus the rest of the patch fixes my_read to wait until socket timeout or until data are available, but at most 0.1 seconds (similar to the code in
my_readline). Of course this is only a work around the caller, which should check the return code and not busy wait.
Regards,
Steffen
| Subject: | Net-HTTP-Methods.patch |
--- /home/work/perl5/perlbrew/perls/perl-5.12.4/lib/site_perl/5.12.4/Net/HTTP/Methods.pm 2012-02-15 20:42:03.000000000 +0100
+++ lib/Net/HTTP/Methods.pm 2012-11-06 20:46:46.622621468 +0100
@@ -233,7 +233,17 @@
return length($_[0]);
}
else {
- return $self->sysread($_[0], $len);
+ { # wait for data, socket might by non-blocking
+ my $n = $self->sysread($_[0], $len);
+ redo if $!{EINTR};
+ if ($!{EAGAIN}) {
+ my $mask = '';
+ vec($mask,$self->fileno,1) = 1;
+ select($mask,undef,undef, ${*$self}{io_socket_timeout} || 0.1);
+ redo;
+ }
+ return $n;
+ }
}
}
}
@@ -513,9 +523,9 @@
my $n = $chunked;
$n = $size if $size && $size < $n;
$n = my_read($self, $$buf_ref, $n);
+ ${*$self}{'http_chunked'} = $chunked - $n||0;
return undef unless defined $n;
- ${*$self}{'http_chunked'} = $chunked - $n;
if ($n > 0) {
if (my $transforms = ${*$self}{'http_te2'}) {