Bug #74431 for Net-HTTP: Net-HTTP chunked decoding bug

Thu Jan 26 13:27:05 2012 bni [...] mail.med.upenn.edu - Ticket created

Subject:	Net-HTTP chunked decoding bug
Date:	Thu, 26 Jan 2012 13:26:35 -0500
To:	bug-Net-HTTP [...] rt.cpan.org
From:	Bryce Nichols <bni [...] mail.med.upenn.edu>

I believe I've found a bug in v6.02 of Net-HTTP near the end of the code for chunked encoding in Methods.pm there is something like: |||$n| |= my_read(||$self||, $||$buf_ref||, ||$n||);| |||return| |undef| |unless| |defined| |$n||;| |||${*||$self||}{||'http_chunked'||} = ||$chunked| |- ||$n||;| The problem is that if the read call gets interrupted then the chunk state is discarded and the next invocation of read_entity_body will fail, since the state no longer matches the position within the stream. I think this may be related to the issue reported in: https://rt.cpan.org/Public/Bug/Display.html?id=72790#txn-1018088 --bryce

Wed May 09 12:11:33 2012 ilmari [...] ilmari.org - Correspondence added

Subject:	[rt.cpan.org #74431] Fix chunked decoding on temporary read error
Date:	Wed, 09 May 2012 17:11:14 +0100
To:	bug-Net-HTTP [...] rt.cpan.org
From:	Dagfinn Ilmari Mannsåker <ilmari [...] ilmari.org>

If the first read after a chunk header returns EAGAIN or EINTR, the chunked state stored in the client object doesn't get updated, so the next call to ->read_entity_body tries to read the chunk header again, and gives a "Missing newline after chunk data" error. This fixes it by updating ${*$self}{'http_chunked'} immediately after the chunk header has been read, so that the state persist to the next call. --- lib/Net/HTTP/Methods.pm | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/lib/Net/HTTP/Methods.pm b/lib/Net/HTTP/Methods.pm index 738e95f..a949b27 100644 --- a/lib/Net/HTTP/Methods.pm +++ b/lib/Net/HTTP/Methods.pm @@ -488,7 +488,7 @@ sub read_entity_body { unless ($chunk_len =~ /^([\da-fA-F]+)\s*$/) { die "Bad chunk-size in HTTP response: $line"; } - $chunked = hex($1); + ${*$self}{'http_chunked'} = $chunked = hex($1); if ($chunked == 0) { ${*$self}{'http_trailers'} = [$self->_read_header_lines]; $$buf_ref = ""; -- 1.7.4.1

Wed May 09 12:11:34 2012 The RT System itself - Status changed from 'new' to 'open'

Fri May 25 11:53:12 2012 rolek [...] bokxing.nl - Correspondence added

From:

rolek [...] bokxing.nl

Show quoted text

> This fixes it by updating ${*$self}{'http_chunked'} immediately after > the chunk header has been read, so that the state persist to the next > call.

I experienced the same bug, with a different effect. When downloading binary files via LWP::UserAgent and https, I would get the complete file without errors, but with 7 extra bytes tacked on at the end. Your patch fixes this and is more elegant than my own solution, so consider this a positive review.

Sat Jun 02 08:34:49 2012 modax [...] debian.org - Correspondence added

From:

modax [...] debian.org

Tre 2012.Gegužė.09 12:11:33 naudotojas ilmari@ilmari.org rašė:

Sat Jun 02 08:40:29 2012 modax [...] debian.org - Correspondence added

From:

modax [...] debian.org

Hello, [ my prior comment was cut short for some reason ] I had a similar problem [1] on Debian unstable which ships 6.03 at the moment. The patch posted here above appears to have fixed it for me. [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=674788

Sun Jun 10 11:07:44 2012 GAAS [...] cpan.org - Correspondence added

Dagfinn's patch was applied as <https://github.com/gisle/net-http/commit/229aaaa2bfb1a06a195ae7d34c0620cfbed530b1>.
Thanks!

Sun Jun 10 11:07:44 2012 GAAS [...] cpan.org - Status changed from 'open' to 'resolved'

Fri Feb 08 18:16:55 2013 ether [...] cpan.org - Fixed in 6.04 added