Skip Menu |

This queue is for tickets about the POE-Component-Client-HTTP CPAN distribution.

Report information
The Basics
Id: 45148
Status: resolved
Priority: 0/
Queue: POE-Component-Client-HTTP

People
Owner: Nobody in particular
Requestors: yikes2000 [...] yahoo.com
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 0.88
Fixed in: (no value)



Subject: HTTPS Broken
When requesting HTTPS URI's, only the first 4 responses are successful. There is a workaround though ... Protocol => 'HTTP/1.0'
Subject: poe_https.pl
#!/usr/bin/perl -w # # PoCo::Client::HTTP ... broken HTTPS # use strict; use Data::Dumper; use HTTP::Request; use POE qw( Component::Client::HTTP ); use constant ME => (0 .. ARG0-1); use constant MAIN_ALIAS => 'main'; use constant UA_ALIAS => 'ua'; use constant STOCKS => qw( AEO AG BBL BCE BMR C CASY CBI CBRL CLP CLWR CNQR EDU EWA EWH FNF GDI HIG HLTH HME IBB JCG KB LTD MFA NNN RMBS SHLD SNH SUSQ TIF TNB UBS VMC YHOO AAPL IBM INTC HP CSCO NVDA BAC GS ); POE::Session->create ( inline_states => { _start => \&start, receive_data => \&receive_data, } ); POE::Kernel->run(); exit; #---------------------------------------- sub start { $_[KERNEL]->alias_set( MAIN_ALIAS ); POE::Component::Client::HTTP->spawn ( Alias => UA_ALIAS, # Protocol => 'HTTP/1.0', Timeout => 10, FollowRedirects => 1, ); foreach my $sym (STOCKS) { request_https_data( @_[(ME)], $sym ); } } sub request_https_data { my $sym = $_[ARG0]; my $url = 'https://moneycentral.msn.com/detail/stock_quote?'. "Symbol=$sym&getquote=Get+Quote"; my $req = HTTP::Request->new( GET => $url ); $req->header( x_sym => $sym ); $_[HEAP]{buf}{$sym} = ''; $_[KERNEL]->post( UA_ALIAS, 'request', 'receive_data', $req ); } sub receive_data { my ($request_packet, $response_packet) = @_[ARG0, ARG1]; my $req_obj = $request_packet->[0]; my $sym = $req_obj->header( 'x_sym' ); my $resp_obj = $response_packet->[0]; my $new_chunk = $resp_obj->decoded_content(); return if ! $new_chunk; my $data = $new_chunk; my @lines = split /<li /, $data; my $lines = @lines + 0; print "<$sym> ".length($data)." bytes; $lines lines\n"; print $data."\n" if $sym =~ /^GS/i; }
I modified your program to tell me how long the requests were taking. You are seeing timeouts because more time has elapsed than your 10-second timeout. <AG> 16904 bytes; 19 lines after 5 seconds <AEO> 34520 bytes; 29 lines after 5 seconds <BCE> 32627 bytes; 28 lines after 5 seconds <BMR> 32359 bytes; 24 lines after 6 seconds <CASY> 34588 bytes; 29 lines after 7 seconds <CBI> 35485 bytes; 29 lines after 9 seconds <CBRL> 34765 bytes; 29 lines after 9 seconds <C> 25720 bytes; 36 lines after 9 seconds <CLP> 32693 bytes; 25 lines after 9 seconds <CLWR> 33575 bytes; 28 lines after 11 seconds ... timeouts begin here ... <EWA> 267 bytes; 1 lines after 12 seconds <EWH> 267 bytes; 1 lines after 12 seconds <FNF> 267 bytes; 1 lines after 12 seconds <GDI> 267 bytes; 1 lines after 12 seconds <HIG> 267 bytes; 1 lines after 12 seconds ... and continue for the remaining ticker symbols. One solution is to increase your timeout. Another is to tune POE::Component::Client::Keepalive to allow more concurrent connections overall, and more concurrent connections per host. See the ConnectionManager parameter for POE::Component::Client::HTTP and the documentation for POE::Component::Client::Keepalive. A third solution is to send a modest number of requests to start with, then send more requsts as they complete. Thank you.