Hi,
I ran into just this issue as recently as yesterday, using
POE::Component::SSLify 1.012 (in fact I just had cpanp update all of
"POE::Component::SSLify POE::Component::SSLify::ClientHandle
POE::Component::SSLify::NonBlock POE::Component::SSLify::NonBlock::ServerHandle
POE::Component::SSLify::ServerHandle" to be sure).
The symptom is that retrieving:
<
https://community.elitedangerous.com/galnet/uid/56a60d089657ba197a730a88>
takes approximately 1 minute, this being the HTTP 1.1 keep-alive timeout
of that server.
I initially thought this to be a bug in POE::Component::Client::HTTP
and thus reported it at:
<
https://rt.cpan.org/Ticket/Display.html?id=111425>
but now I see that the issue is that SSLify doesn't install a custom
driver for retrieving data.
What happens is that POE's loop uses CORE::select() to test if there
is data to be read, but the SSLification of the connection means that
sysread() isn't reading directly from the socket, instead it gets the
plaintext after the SSLification has decrypted it.
In the case of the URL above the returned data is large enough to not
fit in a single 4096 byte read AND the encrypted data is smaller than
the plaintext. So initial sysread() causes the socket to be drained,
and thus CORE::select() to not indicate the file handle as ready for
read, yet not all of the plaintext has actually made it out yet.
This is evidenced by a mixture of activating DEBUG statements in the
POE modules (with some extra code for ISO8601 timestamps) and running it
all under strace to see the actual calls. strace shows read(2) grabbing
over 8302 bytes in the penultimate call on the filehandle BEFORE the
POE code starts hanging pending the server side keep-alive timeout. The
only further read(2) on the socket after this returns zero bytes as
CORE::select() is indicating EOF.
The program then enters a state where the event's
$self->[DRIVER_BOTH]->get() method could still read more, but it isn't
told to because of CORE::select() not saying it's ready to. Once the
connection closes the filehandle is marked as ready for sysread() and
the remainder of the plaintext data is read.
Note that the sysread() calls are triggered by the 'select read'
event being set in POE::Wheel::ReadWrite's _define_read_state, which
calls the event's $self->[DRIVER_BOTH]'s get() method, which in this
case is POE::Driver::SysRW's get().
I'd suggest the real fix is to have SSLify set a custom DRIVER_BOTH
with a get() method that correctly drains the available plaintext as
indicated by CORE::select(). That way the loop in _define_read_state()
will have all available input to process (in this case ultimately by
POE::Filter::HTTPCunk).
I'll see if I can come up with a patch, but not being familiar with
the code I'm not sure I'm capable, or how long it will take.
A workaround for now is to send the header "Connection: close" as part
of the request as this causes the server to not do keep-alive, instead
closing the socket immediately it has sent all of the response. In my
use this is acceptable, but anyone using POE for SSL where they'd like
to make use of keep-alive to reduce overheads will not have this option.
Oh, and I further suspect that this kind of issue is why
POE::Component::Client::HTTP had transparent content decoding (for
compression) disabled as of version 0.84, as the size mismatch between
compressed on-the-wire and plaintext will likely cause the same problem.
So, if a solution can be found for THIS issue then perhaps it's worth
notifying other POE authors.
--
- Athanasius = Athanasius(at)miggy.org /
http://www.miggy.org/
Finger athan(at)fysh.org for PGP key
"And it's me who is my enemy. Me who beats me up.
Me who makes the monsters. Me who strips my confidence." Paula Cole - ME
