Bug #66794 for IO-Socket-IP: Win32 failure (11sockopts.t + 21nonblocking-connect-internet.t)

Tue Mar 22 20:33:13 2011 kmx [...] cpan.org - Ticket created

Subject:

Win32 failure (11sockopts.t + 21nonblocking-connect-internet.t)

Hi,

I have experienced the following failure on my strawberryperl 5.12.2 (32bit) @ MS Windows 7

--
kmx

Running Build test
t\00use.t ........................... ok
t\01local-client-v4.t ............... ok
t\02local-server-v4.t ............... Use of uninitialized value $protocol in socket at D:/strawberry/perl/lib/IO/Socket.pm line 80.
t\02local-server-v4.t ............... 1/26 Use of uninitialized value $protocol in socket at D:/strawberry/perl/lib/IO/Socket.pm line 80.
t\02local-server-v4.t ............... ok
t\03local-cross-v4.t ................ Use of uninitialized value $protocol in socket at D:/strawberry/perl/lib/IO/Socket.pm line 80.
t\03local-cross-v4.t ................ 1/12 Use of uninitialized value $protocol in socket at D:/strawberry/perl/lib/IO/Socket.pm line 80.
t\03local-cross-v4.t ................ ok
t\04local-client-v6.t ............... skipped: No IO::Socket::INET6
t\05local-server-v6.t ............... skipped: No IO::Socket::INET6
t\06local-cross-v6.t ................ Use of uninitialized value $protocol in socket at D:/strawberry/perl/lib/IO/Socket.pm line 80.
t\06local-cross-v6.t ................ 1/12 Use of uninitialized value $protocol in socket at D:/strawberry/perl/lib/IO/Socket.pm line 80.
t\06local-cross-v6.t ................ ok
t\10args.t .......................... ok
t\11sockopts.t ...................... Cannot socket() - at t\11sockopts.t line 11.
Show quoted text

# Looks like your test exited with 255 before it could output anything.

t\11sockopts.t ...................... Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 3/3 subtests
t\12port-fallback.t ................. ok
t\13addrinfo.t ...................... 1/4 Use of uninitialized value $protocol in socket at D:/strawberry/perl/lib/IO/Socket.pm line 80.
t\13addrinfo.t ...................... ok
t\14fileno.t ........................ ok
t\15io-socket.t ..................... Use of uninitialized value $protocol in socket at D:/strawberry/perl/lib/IO/Socket.pm line 80.
t\15io-socket.t ..................... 1/2 Use of uninitialized value $protocol in socket at D:/strawberry/perl/lib/IO/Socket.pm line 80.
t\15io-socket.t ..................... ok
t\20nonblocking-connect.t ........... ok
t\21nonblocking-connect-internet.t .. 1/10
Show quoted text

# Failed test '->connect had to select() at least once'
# at t\21nonblocking-connect-internet.t line 48.

t\21nonblocking-connect-internet.t .. 6/10
Show quoted text

#   Failed test '->connect eventually fails with ECONNREFUSED'
#   at t\21nonblocking-connect-internet.t line 85.
#   dollarbang =

Show quoted text

# Failed test '->connect had to select() at least once'
# at t\21nonblocking-connect-internet.t line 88.

Show quoted text

# Failed test '$socket is not even opened'
# at t\21nonblocking-connect-internet.t line 90.
# Looks like you failed 4 tests of 10.

t\21nonblocking-connect-internet.t .. Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/10 subtests
t\99pod.t ........................... ok

Test Summary Report
-------------------
t\11sockopts.t (Wstat: 65280 Tests: 0 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 3 tests but ran 0.
t\21nonblocking-connect-internet.t (Wstat: 1024 Tests: 10 Failed: 4)
Failed tests: 4, 8-10
Non-zero exit status: 4
Files=16, Tests=109, 12 wallclock secs ( 0.16 usr + 0.08 sys = 0.23 CPU)
Result: FAIL
Failed 2/16 test programs. 4/109 subtests failed.

Wed Apr 20 04:23:50 2011 leonerd-cpan [...] leonerd.org.uk - Correspondence added

On Tue Mar 22 20:33:13 2011, KMX wrote: Show quoted text

> Hi, > > I have experienced the following failure on my strawberryperl

5.12.2 Show quoted text

> (32bit) @ > MS Windows 7

Yes, I am aware. The documentation also explains this part in the BUGS section: * Nonblocking connect fails unit tests on MSWin32 smoke-testing machines. The specifics of the failure are that connect() seems to block anyway despite being asked not to, and that failure to connect is not detected properly. I am as yet unsure why this is. Blocking connect on MSWin32, and both blocking and nonblocking connect on other platforms, all test OK on smoke testing. I'm not sure what's best to suggest here. I don't have a Windows box or really any way to develop and test this myself, I simply note from smoke test failures that it doesn't work. Furthermore, I don't know anything about its exact failure mode or how to fix it. Unless you know of any better ideas, it may be best to "just ignore those test failures". -- Paul Evans

Wed Apr 20 04:23:51 2011 The RT System itself - Status changed from 'new' to 'open'

Wed Apr 20 06:46:31 2011 kmx [...] volny.cz - Correspondence added

CC:	kmx [...] cpan.org
Subject:	Re: [rt.cpan.org #66794] Win32 failure (11sockopts.t + 21nonblocking-connect-internet.t)
Date:	Wed, 20 Apr 2011 12:46:21 +0200
To:	bug-IO-Socket-IP [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> I'm not sure what's best to suggest here. I don't have a Windows > box or really any way to develop and test this myself, I simply > note from smoke test failures that it doesn't work. Furthermore, > I don't know anything about its exact failure mode or how to fix > it. Unless you know of any better ideas, it may be best to "just > ignore those test failures".

Hi Paul, to be honest I am not sure what exactly you are gonna test in 21nonblocking-connect-internet.t thus it is not easy to suggest the fix. Anyway non-blocking socket on Win32 works; however slightly differently than on UNIX systems (AFAIK due to Windows OS internals issues). According to my testing non-blocking $socket->connect() return always 1 (true) on Windows (even if trying to connect to filtered port), therefore while( !$socket->connect and $! == EINPROGRESS ) { ... } will not do a single cycle. I am not saying that this Windows-specific behaviour is right but this is how it works. -- kmx

Thu Apr 21 07:25:44 2011 leonerd-cpan [...] leonerd.org.uk - Correspondence added

On Wed Apr 20 06:46:31 2011, kmx@volny.cz wrote: Show quoted text

> Anyway non-blocking socket on Win32 works; however slightly differently > than on UNIX systems (AFAIK due to Windows OS internals issues). > > According to my testing non-blocking $socket->connect() return always 1 > (true) on Windows (even if trying to connect to filtered port), therefore > > while( !$socket->connect and $! == EINPROGRESS ) { ... } > > will not do a single cycle. > > I am not saying that this Windows-specific behaviour is right but this is > how it works.

Ahhh.. That's highly useful to know. So the ->connect method returns 1.. So how in that case do you know when the connection is ready? Is it still select()able for write-ready? And how can you distinguish an immediate failure? -- Paul Evans

Thu Apr 21 08:52:15 2011 kmx [...] cpan.org - Correspondence added

Show quoted text

> Ahhh.. That's highly useful to know. So the ->connect method returns
> 1.. So how in that case do you know when the
> connection is ready? Is it still select()able for write-ready? And how
> can you distinguish an immediate failure?

The following script:

use strict;
use warnings;

use IO::Socket::IP;
use IO::Select;
use Scalar::Util 'blessed';
use Data::Dumper;

warn "\n###socket1### valid hostname, open tcp port\n";
my $socket1 = IO::Socket::IP->new(
PeerHost    => "cpanidx.org",
PeerService => 80,
Type        => SOCK_STREAM,
Blocking    => 0,
) || warn "cannot create socket1 '$@'\n";

warn "socket1->new retval:\n".Dumper($socket1);
if (blessed($socket1)) {
my $socket = $socket1;
my $r = $socket->connect;
my $c = $socket->connected ? 1 : 0;
warn "socket->connect: rv='$r' \$!='$!' connected='$c'\n";
my $s = IO::Select->new($socket);
warn "socket (can write immediately): " . ($s->can_write(0) ? 1 : 0) . "\n";
warn "socket (can write 1sec timeout): " . ($s->can_write(1) ? 1 : 0) . "\n";
}

warn "\n###socket2### valid hostname, closed/filtered tcp port\n";
my $socket2 = IO::Socket::IP->new(
PeerHost    => "cpanidx.org",
PeerService => 6666,
Type        => SOCK_STREAM,
Blocking    => 0,
) || warn "cannot create socket2 '$@'\n";

warn "socket2->new retval:\n".Dumper($socket2);
if (blessed($socket2)) {
my $socket = $socket2;
my $r = $socket->connect;
my $c = $socket->connected ? 1 : 0;
warn "socket->connect: rv='$r' \$!='$!' connected='$c'\n";
my $s = IO::Select->new($socket);
warn "socket (can write immediately): " . ($s->can_write(0) ? 1 : 0) . "\n";
warn "socket (can write 1sec timeout): " . ($s->can_write(1) ? 1 : 0) . "\n";
}

warn "\n###socket3### invalid hostname, random tcp port\n";
my $socket3 = IO::Socket::IP->new(
PeerHost    => "invalid.host.name.xyz",
PeerService => 6789,
Type        => SOCK_STREAM,
Blocking    => 0,
) || warn "cannot create socket3 '$@'\n";

warn "socket3->new retval:\n".Dumper($socket3);
if (blessed($socket3)) {
my $socket = $socket3;
my $r = $socket->connect;
my $c = $socket->connected ? 1 : 0;
warn "socket->connect: rv='$r' \$!='$!' connected='$c'\n";
my $s = IO::Select->new($socket);
warn "socket (can write immediately): " . ($s->can_write(0) ? 1 : 0) . "\n";
warn "socket (can write 1sec timeout): " . ($s->can_write(1) ? 1 : 0) . "\n";
}

warn "\n###end###\n";

Gives on my Windows7 + strawberry perl 5.12.2 the following output

Show quoted text

###socket1### valid hostname, open tcp port

socket1->new retval:
$VAR1 = bless( \*Symbol::GEN0, 'IO::Socket::IP' );
socket->connect: rv='1' $!='' connected='1'
socket (can write immediately): 0
socket (can write 1sec timeout): 1

Show quoted text

###socket2### valid hostname, closed/filtered tcp port

socket2->new retval:
$VAR1 = bless( \*Symbol::GEN1, 'IO::Socket::IP' );
socket->connect: rv='1' $!='' connected='1'
socket (can write immediately): 0
socket (can write 1sec timeout): 0

Show quoted text

###socket3### invalid hostname, random tcp port

cannot create socket3 'Není znám žádný takový hostitel.'
socket3->new retval:
$VAR1 = 1;

Show quoted text

###end###

--
kmx

Mon Apr 25 13:11:44 2011 leonerd-cpan [...] leonerd.org.uk - Correspondence added

On Thu Apr 21 08:52:15 2011, KMX wrote: Show quoted text

> ###socket1### valid hostname, open tcp port > socket1->new retval: > $VAR1 = bless( \*Symbol::GEN0, 'IO::Socket::IP' ); > socket->connect: rv='1' $!='' connected='1' > socket (can write immediately): 0 > socket (can write 1sec timeout): 1

Ah, that looks useful. Show quoted text

> ###socket2### valid hostname, closed/filtered tcp port > socket2->new retval: > $VAR1 = bless( \*Symbol::GEN1, 'IO::Socket::IP' ); > socket->connect: rv='1' $!='' connected='1' > socket (can write immediately): 0 > socket (can write 1sec timeout): 0

This decidedly less so. :/ How would you determine an error has happened, and if so, which? Perhaps the socket becomes exceptional at that point? Can you test for that as well / instead, in this case? That'd at least find out an error happened. As for knowing which, hrmm.... see the value of $self->getsockopt( SOL_SOCKET, SO_ERROR ); once it returns exceptional state..? -- Paul Evans

Wed Apr 27 03:46:16 2011 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #66794] Win32 failure (11sockopts.t + 21nonblocking-connect-internet.t)
Date:	Wed, 27 Apr 2011 09:46:00 +0200
To:	bug-IO-Socket-IP [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Another test with connection to closed/filtered tcp port (with sockerror code as you have proposed): the code (part): my $socket = IO::Socket::IP->new( PeerHost => "cpanidx.org", PeerService => 6666, Type => SOCK_STREAM, Blocking => 0, ) || warn "cannot create socket '$@'\n"; my $r = $socket->connect; my $c = $socket->connected ? 1 : 0; warn "socket->connect: rv='$r' \$!='$!' connected='$c'\n"; warn "sockerr=".$socket->getsockopt( SOL_SOCKET, SO_ERROR )."\n"; my $s = IO::Select->new($socket); warn "socket (can write immediately): " . ($s->can_write(0) ? 1 : 0) . "\n"; warn "sockerr=".$socket->getsockopt( SOL_SOCKET, SO_ERROR )."\n"; warn "socket (can write 4sec timeout): " . ($s->can_write(4) ? 1 : 0) . "\n"; warn "sockerr=".$socket->getsockopt( SOL_SOCKET, SO_ERROR )."\n"; the output: ###socket2### valid hostname, closed/filtered tcp port socket2->new retval: $VAR1 = bless( \*Symbol::GEN1, 'IO::Socket::IP' ); socket->connect: rv='1' $!='' connected='1' sockerr=0 socket (can write immediately): 0 sockerr=0 socket (can write 4sec timeout): 0 sockerr=10061 Interesting is that if you choose too low timeout in $s->can_write(4) sockerr will be 0 (not 10061). My conclusion for non-blocking connect on Windows 7: 1/ my $socket = IO::Socket::IP->new( ... , Blocking=>0 ) || warn "cannot create socket '$@'\n"; #fails for example if dns name is invalid 2/ $socket->connect; #almost surely returns 1 3/ my $s = IO::Select->new($socket); 4/ $s->can_write(4); #returns an empty array if $socket is not ready for sending data + $socket->getsockopt(SOL_SOCKET, SO_ERROR) will be 10061 in case of connect failure (e.g. closed/filtered port) Just to make it clear: this is how strawberry perl 5.12.2 behaves on my Windows 7 box, I am not saying it is proper behaviour. -- kmx

Thu Oct 13 17:35:07 2011 leonerd-cpan [...] leonerd.org.uk - Correspondence added

I've now put 0.07_001 on CPAN, which may fix this issue. Please re-test and let me know. -- Paul Evans

Sun Oct 16 08:06:20 2011 kmx [...] cpan.org - Correspondence added

Hi Paul,

just a brief feedback - unfortunately t\21nonblocking-connect-internet.t still makes troubles:

C:\home\.cpan\build\IO-Socket-IP-0.07_001-JKvsS1>prove -bv t\21nonblocking-connect-internet.t
t\21nonblocking-connect-internet.t ..
1..10
ok 1 - defined $socket for cpanidx.org:80
ok 2 - $socket not yet connected
ok 3 - ->connect eventually succeeds
ok 4 - ->connect had to select() at least once
ok 5 - $socket now connected
ok 6 - defined $socket for cpanidx.org:6666
ok 7 - $socket not yet connected
select() timed out at t\21nonblocking-connect-internet.t line 84.
Show quoted text

# Looks like you planned 10 tests but ran 7.
# Looks like your test exited with 10035 just after 7.

Dubious, test returned 51 (wstat 13056, 0x3300)
Failed 3/10 subtests

Test Summary Report
-------------------
t\21nonblocking-connect-internet.t (Wstat: 13056 Tests: 7 Failed: 0)
Non-zero exit status: 51
Parse errors: Bad plan. You planned 10 tests but ran 7.
Files=1, Tests=7, 61 wallclock secs ( 0.05 usr + 0.03 sys = 0.08 CPU)
Result: FAIL

--
kmx

Mon Oct 17 12:12:04 2011 leonerd-cpan [...] leonerd.org.uk - Correspondence added

On Sun Oct 16 08:06:20 2011, KMX wrote: Show quoted text

> Hi Paul, > > just a brief feedback - unfortunately t\21nonblocking-connect- > internet.t still > makes troubles:

Hmm yes; as the smokers also observed. This appears to be because MSWin32 doesn't mark a connect() failure as writeready in select(), only as exceptional. So this needs special handling. OK; try 0.07_002: http://search.cpan.org/~pevans/IO-Socket-IP-0.07_002/ If that gets it I'll bump it to 0.08, then see if I can think of a nicer API for it. -- Paul Evans

Mon Oct 17 15:19:11 2011 kmx [...] cpan.org - Correspondence added

Show quoted text

> Hmm yes; as the smokers also observed. This appears to be because
> MSWin32 doesn't mark a connect() failure as writeready in select(), only
> as exceptional. So this needs special handling.

I have tested 0.07_002 on MS Windows + perl 5.14.2 and it pass all tests! I have tested both 32/64bit builds (my hand made builds via gcc)

So it might be a bug in perl 5.12.3@Win32 or something like that.

I would suggest to wait for more cpantesters report for 0.07_002 and then to decide what to do next.

--
kmx

Show quoted text

>
> OK; try 0.07_002:
>
> http://search.cpan.org/~pevans/IO-Socket-IP-0.07_002/
>
> If that gets it I'll bump it to 0.08, then see if I can think of a nicer
> API for it.

Thu Dec 01 11:34:41 2011 leonerd-cpan [...] leonerd.org.uk - Correspondence added

I do believe it is now working, as of 0.07_002. http://www.cpantesters.org/cpan/report/ad96a937-6c0c-1014-9e66- 7776a94ceed8 Have now released _005, which hopefully soon will become a real 0.08 release. -- Paul Evans

Thu Dec 01 11:34:42 2011 leonerd-cpan [...] leonerd.org.uk - Status changed from 'open' to 'patched'

Wed Feb 08 09:33:05 2012 leonerd-cpan [...] leonerd.org.uk - Correspondence added

This appears to be working now: http://matrix.cpantesters.org/?dist=IO-Socket-IP%200.08;reports=1;os=mswin32;perl=5.14.2 -- Paul Evans

Wed Feb 08 09:33:08 2012 The RT System itself - Status changed from 'patched' to 'open'

Wed Feb 08 09:33:08 2012 leonerd-cpan [...] leonerd.org.uk - Status changed from 'open' to 'resolved'

Wed Feb 08 09:33:23 2012 leonerd-cpan [...] leonerd.org.uk - Fixed in 0.07_005 added

Wed Feb 08 09:33:23 2012 leonerd-cpan [...] leonerd.org.uk - Fixed in 0.08 added