Skip Menu |

This queue is for tickets about the POE-Component-Client-HTTP CPAN distribution.

Report information
The Basics
Id: 26874
Status: resolved
Priority: 0/
Queue: POE-Component-Client-HTTP
People
Owner: Nobody in particular
Requestors: KAZUHO [...] cpan.org
Cc:
AdminCc:
Bug Information
Severity: Normal
Broken in: 0.82
Fixed in: (no value)

History Show all quoted text
  Tue May 01 17:05:52 2007 KAZUHO [...] cpan.org - Ticket created
Subject: "incomplete response b" error when using MaxSize option
It seems that POE::Component::Client::HTTP::Request::add_eof erronously generates a 400 "incomplete response b" error when all of the following conditions are met. 1) MaxSize options is used 2) response includes a Content-Length header 3) Content-Length > MaxSize For detail, please see the attached patch.
Subject: PoCoCl-HTTP-Request.pm.MaxSize.patch
--- /usr/local/lib/perl5/site_perl/5.8.8/POE/Component/Client/HTTP/Request.pm Sun Feb 4 14:25:29 2007 +++ POE/Component/Client/HTTP/Request.pm Wed May 2 05:50:50 2007 @@ -12,7 +12,7 @@ # Unique request ID, independent of wheel and timer IDs. my $request_seq = 0; -use constant DEBUG => 0; +use constant DEBUG => 1; use constant REQ_ID => 0; use constant REQ_POSTBACK => 1; @@ -218,11 +218,16 @@ # ignore content for HEAD requests. This may thwart keep-alive, # however. + my $expected = $self->[REQ_FACTORY]->max_response_size; + $expected = $self->[REQ_RESPONSE]->content_length + if + not defined $expected or + $self->[REQ_RESPONSE]->content_length < $expected; if ( $self->[REQ_REQUEST]->method() ne "HEAD" and defined $self->[REQ_RESPONSE]->content_length and not defined $self->[REQ_RESPONSE]->header("Transfer-Encoding") and - $self->[REQ_OCTETS_GOT] < $self->[REQ_RESPONSE]->content_length + $self->[REQ_OCTETS_GOT] < $expected ) { DEBUG and warn( "got " . $self->[REQ_OCTETS_GOT] . " of " .
  Thu Feb 19 01:51:52 2009 RCAPUTO [...] cpan.org - Correspondence added
On Tue May 01 17:05:52 2007, KAZUHO wrote: Show quoted text
> It seems that POE::Component::Client::HTTP::Request::add_eof erronously > generates a 400 "incomplete response b" error when all of the following > conditions are met. > > 1) MaxSize options is used > 2) response includes a Content-Length header > 3) Content-Length > MaxSize > > For detail, please see the attached patch.
Hi, KAZUHO. I appreciate the test case. It definitely fails for me here. And I also appreciate the patch, but applying it causes several tests to fail. Can you reformulate your patch so that "make test" passes? Thank you.
  Thu Feb 19 01:51:53 2009 The RT System itself - Status changed from 'new' to 'open'
  Tue Feb 24 03:43:01 2009 perl [...] evancarroll.com - Correspondence added
Another issue, this error seems to hold precedence over more important server error (400s). Which is why i'm failing t/01
  Sat Jun 13 23:55:21 2009 ian [...] dal-acm.ca - Correspondence added
From: ian [...] dal-acm.ca
On Thu Feb 19 01:51:52 2009, RCAPUTO wrote: Show quoted text
> On Tue May 01 17:05:52 2007, KAZUHO wrote:
> > It seems that POE::Component::Client::HTTP::Request::add_eof erronously > > generates a 400 "incomplete response b" error when all of the following > > conditions are met. > > > > 1) MaxSize options is used > > 2) response includes a Content-Length header > > 3) Content-Length > MaxSize > > > > For detail, please see the attached patch.
> > Hi, KAZUHO. I appreciate the test case. It definitely fails for me > here. And I also appreciate the patch, but applying it causes several > tests to fail. > > Can you reformulate your patch so that "make test" passes? Thank you.
There are some issues with the provided patch, but is the test it's failing (#9 for me) even correct? Perhaps I'm reading it wrong, but it sets a content length of 300, then gives a body of 300... and expects a 400 error because the content length is too long.
  Mon Jul 13 20:55:44 2009 bitcard [...] ew.ewheeler.org - Correspondence added
From: bitcard [...] ew.ewheeler.org
Show quoted text
> > On Tue May 01 17:05:52 2007, KAZUHO wrote:
> > > It seems that POE::Component::Client::HTTP::Request::add_eof
> erronously
> > > generates a 400 "incomplete response b" error when all of the
> following
> > > conditions are met. > > > > > > 1) MaxSize options is used > > > 2) response includes a Content-Length header > > > 3) Content-Length > MaxSize
I've found the same problem shipping to UPS's shipping estimation interface. Attached is a short example script that reliably regenerates the example. Also attached is the actual data being posted in case you need it (see $postdata in the script). I tried applying KAZUHO's patch, and this did not fix my problem. In my case, this POST works with HTTP, but not HTTPS (w/ SSLify installed). Note that the response from UPS does not end with a trailing carriage return. Is something buffered that should be flushed that isn't?
#!/usr/bin/perl # My code manifested the same error as this bug report: # http://rt.cpan.org/Public/Bug/Display.html?id=26874 # This is an attempt to supply a test case for the bug report. use warnings; use strict; use lib qw(. ../../libs ../../libs/lib/perl5); use POE qw/ Component::Client::HTTP Wheel::ListenAccept/; use HTTP::Request; use constant TIMEOUT => 15; POE::Session->create( inline_states => { _start => sub { POE::Component::Client::HTTP->spawn( Agent => 'SpiffCrawler/0.90', # defaults to something long Alias => 'UserAgent', # defaults to 'weeble' Timeout => TIMEOUT, # defaults to 180 seconds MaxSize => 1024*1024, # defaults to entire response ); # Retrieve the data to post to UPS. my $postdata = `wget -O- http://code.ewheeler.org/ssi/ups-request.xml`; # This problem only manifests with an https connection. # The http connection works fine. It acts as if not all data # is flushed in the response. Note that UPS does not feed us a CR/LF. #my $req = HTTP::Request->new('POST', "http://wwwcie.ups.com//ups.app/xml/Rate"); my $req = HTTP::Request->new('POST', "https://wwwcie.ups.com//ups.app/xml/Rate"); $req->content($postdata); $_[KERNEL]->post( 'UserAgent' => 'request', 'UserAgent_response', $req, ); $_[KERNEL]->post( 'UserAgent' => 'request', 'UserAgent_response', $req, ); print "Data posted...waiting for response or timeout " . TIMEOUT . "s\n" }, UserAgent_response => sub { my ($request_packet, $response_packet, $arg2, $arg3) = @_[ARG0, ARG1]; my $req = $request_packet->[0]; my $tag = $request_packet->[1]; my $response = $response_packet->[0]; print("tag=$tag\n" . "*" x 78, "\n", "*** my request:\n", "-" x 78, "\n", $req->as_string(), "*" x 78, "\n", "*** their response:\n", "-" x 78, "\n", $response->as_string(), ); } } ); POE::Kernel->run(); exit 0;
<?xml version='1.0' standalone='yes'?> <AccessRequest> <AccessLicenseNumber>1C44B1E71BB78164</AccessLicenseNumber> <Password>anypass</Password> <UserId>anyuser</UserId> </AccessRequest> <?xml version='1.0' standalone='yes'?> <RatingServiceSelectionRequest> <PickupType> <Code>06</Code> </PickupType> <Request> <RequestAction>Rate</RequestAction> <RequestOption>shop</RequestOption> <TransactionReference> <CustomerContext>Net::UPS</CustomerContext> <XpciVersion>1.0001</XpciVersion> </TransactionReference> </Request> <Shipment> <Package> <DimensionalWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> </DimensionalWeight> <Dimensions> <Height>10</Height> <Length>10</Length> <UnitOfMeasurement> <Code>IN</Code> </UnitOfMeasurement> <Width>10</Width> </Dimensions> <PackageWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> <Weight>27</Weight> </PackageWeight> <PackagingType> <Code>02</Code> </PackagingType> </Package> <Package> <DimensionalWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> </DimensionalWeight> <Dimensions> <Height>10</Height> <Length>10</Length> <UnitOfMeasurement> <Code>IN</Code> </UnitOfMeasurement> <Width>10</Width> </Dimensions> <PackageWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> <Weight>27</Weight> </PackageWeight> <PackagingType> <Code>02</Code> </PackagingType> </Package> <Package> <DimensionalWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> </DimensionalWeight> <Dimensions> <Height>10</Height> <Length>10</Length> <UnitOfMeasurement> <Code>IN</Code> </UnitOfMeasurement> <Width>10</Width> </Dimensions> <PackageWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> <Weight>27</Weight> </PackageWeight> <PackagingType> <Code>02</Code> </PackagingType> </Package> <Package> <DimensionalWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> </DimensionalWeight> <Dimensions> <Height>10</Height> <Length>10</Length> <UnitOfMeasurement> <Code>IN</Code> </UnitOfMeasurement> <Width>10</Width> </Dimensions> <PackageWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> <Weight>27</Weight> </PackageWeight> <PackagingType> <Code>02</Code> </PackagingType> </Package> <Package> <DimensionalWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> </DimensionalWeight> <Dimensions> <Height>10</Height> <Length>10</Length> <UnitOfMeasurement> <Code>IN</Code> </UnitOfMeasurement> <Width>10</Width> </Dimensions> <PackageWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> <Weight>27</Weight> </PackageWeight> <PackagingType> <Code>02</Code> </PackagingType> </Package> <Service> <Code>03</Code> </Service> <ShipTo> <Address> <CountryCode>US</CountryCode> <PostalCode>97007</PostalCode> </Address> </ShipTo> <Shipper> <Address> <CountryCode>US</CountryCode> <PostalCode>91789</PostalCode> </Address> </Shipper> </Shipment> </RatingServiceSelectionRequest>
  Sun Jul 26 13:38:02 2009 RCAPUTO [...] cpan.org - Correspondence added
On Mon Jul 13 20:55:44 2009, theg33kb0y wrote: Show quoted text
> > > On Tue May 01 17:05:52 2007, KAZUHO wrote:
> > > > It seems that POE::Component::Client::HTTP::Request::add_eof
> > erronously
> > > > generates a 400 "incomplete response b" error when all of the
> > following
> > > > conditions are met. > > > > > > > > 1) MaxSize options is used > > > > 2) response includes a Content-Length header > > > > 3) Content-Length > MaxSize
> > I've found the same problem shipping to UPS's shipping estimation > interface. Attached is a short example script that reliably regenerates > the example. Also attached is the actual data being posted in case you > need it (see $postdata in the script). > > I tried applying KAZUHO's patch, and this did not fix my problem. In my > case, this POST works with HTTP, but not HTTPS (w/ SSLify installed). > > Note that the response from UPS does not end with a trailing carriage > return. Is something buffered that should be flushed that isn't?
It appears that something has changed in the past fortnight. This test program and data don't trigger the bug. :( I'll try to craft a test case that doesn't require an external web server.
  Sun Jul 26 19:02:46 2009 RCAPUTO [...] cpan.org - Correspondence added
I have resolved this in revision 365: If the Content-Length is greater than MaxSize, or the length of the content is (in the case where there is no Content-Length header), then we return a 406 Not Acceptable error rather than truncated data. If you prefer the truncated data behavior, please use a range request instead of MaxSize. The range request will also let you complete the file if you want.
  Sun Jul 26 19:02:48 2009 RCAPUTO [...] cpan.org - Status changed from 'open' to 'resolved'