Bug #44569 for LWPx-ParanoidAgent: [PATCH] send_request has to run the response_done handler for mirror() otherwise the mirrored file could remain truncated (added also a test script which must be run manually)

Wed Mar 25 13:44:57 2009 srezic [...] iconmobile.com - Ticket created

CC:	Slaven Rezic <srezic [...] iconmobile.com>
Subject:	[PATCH] send_request has to run the response_done handler for mirror() otherwise the mirrored file could remain truncated (added also a test script which must be run manually)
Date:	Wed, 25 Mar 2009 18:37:04 +0100
To:	bug-LWPx-ParanoidAgent [...] rt.cpan.org
From:	Slaven Rezic <srezic [...] iconmobile.com>

--- MANIFEST | 1 + lib/LWPx/ParanoidAgent.pm | 1 + t/30-mirror.t | 27 +++++++++++++++++++++++++++ 3 files changed, 29 insertions(+), 0 deletions(-) create mode 100755 t/30-mirror.t diff --git a/MANIFEST b/MANIFEST index d6a2083..d2fecb6 100644 --- a/MANIFEST +++ b/MANIFEST @@ -5,4 +5,5 @@ lib/LWPx/Protocol/http_paranoid.pm lib/LWPx/Protocol/https_paranoid.pm lib/LWPx/ParanoidAgent.pm t/00-all.t +t/30-mirror.t META.yml Module meta-data (added by MakeMaker) diff --git a/lib/LWPx/ParanoidAgent.pm b/lib/LWPx/ParanoidAgent.pm index 0db4380..ae77419 100644 --- a/lib/LWPx/ParanoidAgent.pm +++ b/lib/LWPx/ParanoidAgent.pm @@ -322,6 +322,7 @@ EOT $response->request($request); # record request for reference $cookie_jar->extract_cookies($response) if $cookie_jar; $response->header("Client-Date" => HTTP::Date::time2str(time)); + $self->run_handlers("response_done", $response); return $response; } diff --git a/t/30-mirror.t b/t/30-mirror.t new file mode 100755 index 0000000..9f0a3ea --- /dev/null +++ b/t/30-mirror.t @@ -0,0 +1,27 @@ +# Testing the "Transfer truncated: only ... out of .. bytes received" +# case. + +use strict; +use Test::More; + +use File::Temp qw(tempfile); +use Getopt::Long qw(GetOptions); +use LWPx::ParanoidAgent; + +my $url; +GetOptions("url=s" => \$url) + or die "usage: $0 [-url url]"; + +if (!$url) { + plan skip_all => 'Mirror tests needs -url option'; + exit; +} + +plan tests => 1; + +my(undef, $tempfile) = tempfile(UNLINK => 1); +unlink $tempfile; # we only need the filename +my $ua = LWPx::ParanoidAgent->new; +my $resp = $ua->mirror($url, $tempfile); +ok($resp->is_success) + or diag($resp->as_string); -- 1.6.2

Wed Mar 25 13:47:49 2009 SREZIC [...] cpan.org - Correspondence added

Same as before, but now as an attachment. To reproduce the problem, a fresh libwww-perl is needed and then just try perl -Mblib t/30-mirror.t http://someurl Without the patch, it will say something like Transfer truncated: only 4096 out of 7582 bytes received Regards, Slaven

From 92e3e2cfef2a67bace93890d10f104ec9db4bf18 Mon Sep 17 00:00:00 2001 From: Slaven Rezic <srezic@iconmobile.com> Date: Wed, 25 Mar 2009 18:28:06 +0100 Subject: [PATCH] send_request has to run the response_done handler for mirror() otherwise the mirrored file could remain truncated (added also a test script which must be run manually) --- MANIFEST | 1 + lib/LWPx/ParanoidAgent.pm | 1 + t/30-mirror.t | 27 +++++++++++++++++++++++++++ 3 files changed, 29 insertions(+), 0 deletions(-) create mode 100755 t/30-mirror.t diff --git a/MANIFEST b/MANIFEST index d6a2083..d2fecb6 100644 --- a/MANIFEST +++ b/MANIFEST @@ -5,4 +5,5 @@ lib/LWPx/Protocol/http_paranoid.pm lib/LWPx/Protocol/https_paranoid.pm lib/LWPx/ParanoidAgent.pm t/00-all.t +t/30-mirror.t META.yml Module meta-data (added by MakeMaker) diff --git a/lib/LWPx/ParanoidAgent.pm b/lib/LWPx/ParanoidAgent.pm index 0db4380..ae77419 100644 --- a/lib/LWPx/ParanoidAgent.pm +++ b/lib/LWPx/ParanoidAgent.pm @@ -322,6 +322,7 @@ EOT $response->request($request); # record request for reference $cookie_jar->extract_cookies($response) if $cookie_jar; $response->header("Client-Date" => HTTP::Date::time2str(time)); + $self->run_handlers("response_done", $response); return $response; } diff --git a/t/30-mirror.t b/t/30-mirror.t new file mode 100755 index 0000000..9f0a3ea --- /dev/null +++ b/t/30-mirror.t @@ -0,0 +1,27 @@ +# Testing the "Transfer truncated: only ... out of .. bytes received" +# case. + +use strict; +use Test::More; + +use File::Temp qw(tempfile); +use Getopt::Long qw(GetOptions); +use LWPx::ParanoidAgent; + +my $url; +GetOptions("url=s" => \$url) + or die "usage: $0 [-url url]"; + +if (!$url) { + plan skip_all => 'Mirror tests needs -url option'; + exit; +} + +plan tests => 1; + +my(undef, $tempfile) = tempfile(UNLINK => 1); +unlink $tempfile; # we only need the filename +my $ua = LWPx::ParanoidAgent->new; +my $resp = $ua->mirror($url, $tempfile); +ok($resp->is_success) + or diag($resp->as_string); -- 1.6.2

Wed Mar 25 13:47:50 2009 The RT System itself - Status changed from 'new' to 'open'

Wed Mar 25 13:55:11 2009 SREZIC [...] cpan.org - Cc SREZIC added

Tue Mar 09 12:11:19 2010 SREZIC [...] cpan.org - Correspondence added

On Wed Mar 25 13:47:49 2009, SREZIC wrote: Show quoted text

> Same as before, but now as an attachment. > > To reproduce the problem, a fresh libwww-perl is needed and then just try > > perl -Mblib t/30-mirror.t http://someurl > > Without the patch, it will say something like > > Transfer truncated: only 4096 out of 7582 bytes received >

For some reason, it does not currently fail for every URL: $ perl -Mblib t/30-mirror.t -url http://www.google.com 1..1 ok 1 $ perl -Mblib t/30-mirror.t -url http://search.cpan.org 1..1 Transfer truncated: only 4096 out of 4890 bytes received # Looks like your test exited with 255 before it could output anything. Maybe only if some size limit (4096?) is exceeded? Regards, Slaven

Wed Mar 10 04:56:20 2010 SREZIC [...] cpan.org - Correspondence added

Attached an amended patch which works without fuzziness against 1.07, and which also does not fail with older LWP anymore. Regards, Slaven

Subject:

LWPx-ParanoidAgent-1.07-SREZIC-01.patch

diff --git a/MANIFEST b/MANIFEST index d6a2083..d2fecb6 100644 --- a/MANIFEST +++ b/MANIFEST @@ -5,4 +5,5 @@ lib/LWPx/Protocol/http_paranoid.pm lib/LWPx/Protocol/https_paranoid.pm lib/LWPx/ParanoidAgent.pm t/00-all.t +t/30-mirror.t META.yml Module meta-data (added by MakeMaker) diff --git a/lib/LWPx/ParanoidAgent.pm b/lib/LWPx/ParanoidAgent.pm index 7409e77..97b0642 100644 --- a/lib/LWPx/ParanoidAgent.pm +++ b/lib/LWPx/ParanoidAgent.pm @@ -331,6 +331,7 @@ EOT $response->request($request); # record request for reference $cookie_jar->extract_cookies($response) if $cookie_jar; $response->header("Client-Date" => HTTP::Date::time2str(time)); + $self->run_handlers("response_done", $response) if $self->can('run_handlers'); return $response; } diff --git a/t/30-mirror.t b/t/30-mirror.t new file mode 100755 index 0000000..9f0a3ea --- /dev/null +++ b/t/30-mirror.t @@ -0,0 +1,27 @@ +# Testing the "Transfer truncated: only ... out of .. bytes received" +# case. + +use strict; +use Test::More; + +use File::Temp qw(tempfile); +use Getopt::Long qw(GetOptions); +use LWPx::ParanoidAgent; + +my $url; +GetOptions("url=s" => \$url) + or die "usage: $0 [-url url]"; + +if (!$url) { + plan skip_all => 'Mirror tests needs -url option'; + exit; +} + +plan tests => 1; + +my(undef, $tempfile) = tempfile(UNLINK => 1); +unlink $tempfile; # we only need the filename +my $ua = LWPx::ParanoidAgent->new; +my $resp = $ua->mirror($url, $tempfile); +ok($resp->is_success) + or diag($resp->as_string);

Tue Mar 19 10:53:34 2013 cpan [...] barely3am.com - Correspondence added

https://github.com/collectiveintel/LWPx- ParanoidAgent/commit/a9d5d7d4f2b7c86b0ecfc66efd60c749582b2001 haven't tested yet.. will soon. i have co-maintainer rights, so we can get some of these pushed out.

Mon Nov 04 14:28:34 2013 cpan [...] barely3am.com - Status changed from 'open' to 'resolved'

Bug #44569 for LWPx-ParanoidAgent: [PATCH] send_request has to run the response_done handler for mirror() otherwise the mirrored file could remain truncated (added also a test script which must be run manually)

Preferred bug tracker