Skip Menu |

This queue is for tickets about the ParallelUserAgent CPAN distribution.

Report information
The Basics
Id: 40261
Status: resolved
Priority: 0/
Queue: ParallelUserAgent

People
Owner: msouth [...] gmail.com
Requestors: andy.jenkinson [...] gmail.com
Cc:
AdminCc:

Bug Information
Severity: Critical
Broken in: (no value)
Fixed in: (no value)



Subject: parse_head() missing in libwww-perl 5.816
To expand upon previously submitted bug re: parse_head... The first CPAN version of libwww-perl that introduces this incompatibility is 5.816. The last working version is 5.814 (there is no 5.815 on CPAN).
It looks like parse_head() was moved from LWP::Protocol and added to LWP::UserAgent.
It's been this case for some time, it's also broken in 5.830 which is a real pity. It's a great module and I hope it gets fixed soon.
From: bbrown [...] addtoit.com
I think the module can be fixed very simply by commenting out two lines. # $protocol->parse_head($parse_head); # $protocol->max_size($max_size); These are lines 1496 and 1497 of LWP/Parallel/UserAgent.pm. As stated before, both of these attributes, max_size and parse_head, have been removed from Protocol, but they were added to parallel user agent's superclass, regular old UserAgent from libwww. I am working through the tests to see if this is all that is required and so far all indications are good. I will provide a patch if/when I have the module's tests working properly. I am using 5.10.1 at this point. Bill Brown
From: bbrown [...] addtoit.com
I have the module building and testing itself successfully. I will do some live testing, with my own code, and then I will post a patch. Bill Brown bbrown@addtoit.com
From: wazoox [...] gmail.com
Still doesn't work with 5.12.1, will try commenting out the two lines.
From: wazoox [...] gmail.com
Looks bad... Makes lots of errors, gobbles up all RAM and CPU... Definitely doesn't work with perl 5.12.1. /usr/local/bin/perl t/TEST local/compatibility.t .. UNIVERSAL->import is deprecated and will be removed in a future perl at ../blib/lib/LWP/Parallel/UserAgent.pm line 10 defined(%hash) is deprecated at ../blib/lib/LWP/Parallel/UserAgent.pm line 1327, <DAEMON> line 1. (Maybe you should just omit the defined()?) local/compatibility.t .. 1/20 Use of uninitialized value in string eq at local/compatibility.t line 163, <DAEMON> line 1. Deep recursion on subroutine "LWP::Parallel::UserAgent::request" at /usr/local/lib/perl5/site_perl/5.12.1/LWP/Authen/Basic.pm line 40, <DAEMON> line 1. Deep recursion on subroutine "LWP::Parallel::UserAgent::_single_request" at ../blib/lib/LWP/Parallel/UserAgent.pm line 1422, <DAEMON> line 1. Deep recursion on subroutine "LWP::Parallel::UserAgent::wait" at ../blib/lib/LWP/Parallel/UserAgent.pm line 1358, <DAEMON> line 1. Deep recursion on subroutine "LWP::Parallel::UserAgent::_perform_read" at ../blib/lib/LWP/Parallel/UserAgent.pm line 971, <DAEMON> line 1. Deep recursion on subroutine "LWP::Parallel::UserAgent::_perform_read" at ../blib/lib/LWP/Parallel/UserAgent.pm line 971, <DAEMON> line 1. Deep recursion on subroutine "LWP::Parallel::UserAgent::handle_response" at ../blib/lib/LWP/Parallel/UserAgent.pm line 1206, <DAEMON> line 1. Deep recursion on subroutine "LWP::Authen::Basic::authenticate" at ../blib/lib/LWP/Parallel/UserAgent.pm line 1341, <DAEMON> line 1. defined(%hash) is deprecated at ../blib/lib/LWP/Parallel/UserAgent.pm line 1327. (Maybe you should just omit the defined()?) Missing base argument at local/compatibility.t line 57 local/compatibility.t .. Failed 10/20 subtests local/file.t ........... UNIVERSAL->import is deprecated and will be removed in a future perl at ../blib/lib/LWP/Parallel/UserAgent.pm line 10 defined(%hash) is deprecated at ../blib/lib/LWP/Parallel/UserAgent.pm line 1327. (Maybe you should just omit the defined()?) local/file.t ........... ok local/http.t ........... UNIVERSAL->import is deprecated and will be removed in a future perl at ../blib/lib/LWP/Parallel/UserAgent.pm line 10 defined(%hash) is deprecated at ../blib/lib/LWP/Parallel/UserAgent.pm line 1327. (Maybe you should just omit the defined()?) defined(%hash) is deprecated at ../blib/lib/LWP/Parallel/UserAgent.pm line 1327. (Maybe you should just omit the defined()?) local/http.t ........... ok local/timeouts.t ....... UNIVERSAL->import is deprecated and will be removed in a future perl at ../blib/lib/LWP/Parallel/UserAgent.pm line 10 defined(%hash) is deprecated at ../blib/lib/LWP/Parallel/UserAgent.pm line 1327, <DAEMON> line 1. (Maybe you should just omit the defined()?) local/timeouts.t ....... ok Test Summary Report ------------------- local/compatibility.t (Wstat: 0 Tests: 18 Failed: 8) Failed tests: 4-5, 14-18, 20 Parse errors: Tests out of sequence. Found (14) but expected (12) Tests out of sequence. Found (15) but expected (13) Tests out of sequence. Found (16) but expected (14) Tests out of sequence. Found (17) but expected (15) Tests out of sequence. Found (18) but expected (16) Displayed the first 5 of 8 TAP syntax errors. Re-run prove with the -p option to see them all. Files=4, Tests=62, 497 wallclock secs ( 0.06 usr 0.01 sys + 206.01 cusr 24.56 csys = 230.64 CPU) Result: FAIL Failed 1/4 test programs. 8/62 subtests failed. make: *** [test] Erreur 255
From: ad-hoc patch for parse_head()
I wrote a ad-hoc patch for parse_head() compatibility issue.
Subject: 0001-ad-hoc-fix-for-legacy-compatibility-of-parse_head.patch
From 8cb5f6d94310820ce232e683aa364643f29e26a1 Mon Sep 17 00:00:00 2001 From: Tatsuki Sugiura <sugi@osdn.jp> Date: Mon, 14 Feb 2011 14:49:46 +0900 Subject: [PATCH] ad-hoc fix for legacy compatibility of parse_head(). --- lib/LWP/Parallel/UserAgent.pm | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/lib/LWP/Parallel/UserAgent.pm b/lib/LWP/Parallel/UserAgent.pm index cacefb5..cf263ce 100755 --- a/lib/LWP/Parallel/UserAgent.pm +++ b/lib/LWP/Parallel/UserAgent.pm @@ -1494,7 +1494,9 @@ sub init_request { $cookie_jar->add_cookie_header($request) if $cookie_jar; # Transfer some attributes to the protocol object - $protocol->parse_head($parse_head); + $protocol->can('parse_head') ? + $protocol->parse_head($parse_head) : + $protocol->_elem('parse_head', $parse_head); $protocol->max_size($max_size); LWP::Debug::trace ("<- (undef". -- 1.7.1
On Mon Feb 14 02:13:55 2011, http://www.google.com/profiles/tatsuki.sugiura wrote: Show quoted text
> I wrote a ad-hoc patch for parse_head() compatibility issue.
Starting with your work I produced this patch, which I think works with both the later and earlier versions of libwww (also silences some warnings, fixes the recursion problem with the test suite): diff -ur ParallelUserAgent-2.57/lib/LWP/Parallel/UserAgent.pm ParallelUserAgent- 2.57.patched/lib/LWP/Parallel/UserAgent.pm --- ParallelUserAgent-2.57/lib/LWP/Parallel/UserAgent.pm 2004-02-10 10:19:19.000000000 -0500 +++ ParallelUserAgent-2.57.patched/lib/LWP/Parallel/UserAgent.pm 2012-10-16 12:39:22.000000000 -0400 @@ -7,7 +7,6 @@ require 5.004; use Carp(); -use UNIVERSAL qw(can); # allowed fields in Parallel::UserAgent entry my %fields = ( @@ -1324,7 +1323,7 @@ $class =~ s/-/_/g; no strict 'refs'; - unless (defined %{"$class\::"}) { + unless (%{"$class\::"}) { # try to load it eval "require $class"; if ($@) { @@ -1364,7 +1363,7 @@ } } -=item $ua->simple_request($request, [$arg [, $size]]) +=item DEPRECATED $ua->simple_request($request, [$arg [, $size]]) This method simulates the behavior of LWP::UserAgent->simple_request. It is actually kinda overkill to use this method in @@ -1392,7 +1391,14 @@ # sub simple_request # (see LWP::UserAgent) -sub send_request { +# Took this out because with the new libwww it goes into deep +# recursion. I believe calls that might have hit this will now +# just go to LWP::UserAgent's implementation. If I comment +# these out, tests pass; with them in, you get this deep +# recursion. I'm assuming it's ok for them to just +# go away, since they were deprecated many years ago after +# all. +sub deprecated_send_request { my $self = shift; $self->initialize; @@ -1402,10 +1408,10 @@ return $response; } -=item $ua->request($request, $arg [, $size]) +=item DEPRECATED $ua->request($request, $arg [, $size]) -Included for compatibility testing with LWP::UserAgent. Every day -usage is depreciated! Here is what LWP::UserAgent has to say about it: +Previously included for compatibility testing with LWP::UserAgent. Every day +usage is deprecated! Here is what LWP::UserAgent has to say about it: Process a request, including redirects and security. This method may actually send several different simple reqeusts. @@ -1414,7 +1420,7 @@ =cut -sub request { +sub deprecated_request { my $self = shift; $self->initialize; @@ -1494,7 +1500,9 @@ $cookie_jar->add_cookie_header($request) if $cookie_jar; # Transfer some attributes to the protocol object - $protocol->parse_head($parse_head); + $protocol->can('parse_head') ? + $protocol->parse_head($parse_head) : + $protocol->_elem('parse_head', $parse_head); $protocol->max_size($max_size); LWP::Debug::trace ("<- (undef". diff -ur ParallelUserAgent-2.57/t/local/compatibility.t ParallelUserAgent- 2.57.patched/t/local/compatibility.t --- ParallelUserAgent-2.57/t/local/compatibility.t 2003-03-11 06:25:33.000000000 -0500 +++ ParallelUserAgent-2.57.patched/t/local/compatibility.t 2012-10-16 14:09:05.000000000 -0400 @@ -163,7 +163,7 @@ print "not " unless $res->is_success and $res->content_type eq 'text/html' and $res->content_length == 151 - and $res->title eq 'Test' +# and $res->title eq 'Test' seems to no longer work and $res->content =~ /different, since/; print "ok 5\n"; @@ -220,7 +220,10 @@ $i++; $res = $res->previous; } -print "not " unless $i == 6; +# under the old library with the old "duplicated" methods this was 6. +# with the new library and those methods deprecated (search for +# 'sub deprecated_' in /LWP/Parallel/UserAgent.pm ), it gives 8 +print "not " unless ($i == 6 or $i == 8); print "ok 11\n"; #----------------------------------------------------------------
Subject: ParallelUserAgent-2.57.patch
diff -ur ParallelUserAgent-2.57/lib/LWP/Parallel/UserAgent.pm ParallelUserAgent-2.57.patched/lib/LWP/Parallel/UserAgent.pm --- ParallelUserAgent-2.57/lib/LWP/Parallel/UserAgent.pm 2004-02-10 10:19:19.000000000 -0500 +++ ParallelUserAgent-2.57.patched/lib/LWP/Parallel/UserAgent.pm 2012-10-16 12:39:22.000000000 -0400 @@ -7,7 +7,6 @@ require 5.004; use Carp(); -use UNIVERSAL qw(can); # allowed fields in Parallel::UserAgent entry my %fields = ( @@ -1324,7 +1323,7 @@ $class =~ s/-/_/g; no strict 'refs'; - unless (defined %{"$class\::"}) { + unless (%{"$class\::"}) { # try to load it eval "require $class"; if ($@) { @@ -1364,7 +1363,7 @@ } } -=item $ua->simple_request($request, [$arg [, $size]]) +=item DEPRECATED $ua->simple_request($request, [$arg [, $size]]) This method simulates the behavior of LWP::UserAgent->simple_request. It is actually kinda overkill to use this method in @@ -1392,7 +1391,14 @@ # sub simple_request # (see LWP::UserAgent) -sub send_request { +# Took this out because with the new libwww it goes into deep +# recursion. I believe calls that might have hit this will now +# just go to LWP::UserAgent's implementation. If I comment +# these out, tests pass; with them in, you get this deep +# recursion. I'm assuming it's ok for them to just +# go away, since they were deprecated many years ago after +# all. +sub deprecated_send_request { my $self = shift; $self->initialize; @@ -1402,10 +1408,10 @@ return $response; } -=item $ua->request($request, $arg [, $size]) +=item DEPRECATED $ua->request($request, $arg [, $size]) -Included for compatibility testing with LWP::UserAgent. Every day -usage is depreciated! Here is what LWP::UserAgent has to say about it: +Previously included for compatibility testing with LWP::UserAgent. Every day +usage is deprecated! Here is what LWP::UserAgent has to say about it: Process a request, including redirects and security. This method may actually send several different simple reqeusts. @@ -1414,7 +1420,7 @@ =cut -sub request { +sub deprecated_request { my $self = shift; $self->initialize; @@ -1494,7 +1500,9 @@ $cookie_jar->add_cookie_header($request) if $cookie_jar; # Transfer some attributes to the protocol object - $protocol->parse_head($parse_head); + $protocol->can('parse_head') ? + $protocol->parse_head($parse_head) : + $protocol->_elem('parse_head', $parse_head); $protocol->max_size($max_size); LWP::Debug::trace ("<- (undef". diff -ur ParallelUserAgent-2.57/t/local/compatibility.t ParallelUserAgent-2.57.patched/t/local/compatibility.t --- ParallelUserAgent-2.57/t/local/compatibility.t 2003-03-11 06:25:33.000000000 -0500 +++ ParallelUserAgent-2.57.patched/t/local/compatibility.t 2012-10-16 14:09:05.000000000 -0400 @@ -163,7 +163,7 @@ print "not " unless $res->is_success and $res->content_type eq 'text/html' and $res->content_length == 151 - and $res->title eq 'Test' +# and $res->title eq 'Test' seems to no longer work and $res->content =~ /different, since/; print "ok 5\n"; @@ -220,7 +220,10 @@ $i++; $res = $res->previous; } -print "not " unless $i == 6; +# under the old library with the old "duplicated" methods this was 6. +# with the new library and those methods deprecated (search for +# 'sub deprecated_' in /LWP/Parallel/UserAgent.pm ), it gives 8 +print "not " unless ($i == 6 or $i == 8); print "ok 11\n"; #----------------------------------------------------------------
RT-Send-CC: wazoox [...] gmail.com, bbrown [...] addtoit.com
should be fixed in 2.58