Bug #104759 for IO-Socket-SSL: Can't connect to www.microsoft.com:443 -- certificate verify failed

Thu May 28 18:04:45 2015 GAAS [...] cpan.org - Ticket created

Subject:

Can't connect to www.microsoft.com:443 -- certificate verify failed

The following test program demonstrates:

use IO::Socket::SSL; # I'm using 2.015

my @hosts = qw(

www.apple.com

www.microsoft.com

);

print "1..", int(@hosts), "\n";

my $testnum = 0;

for my $h (@hosts) {

print "# $h\n";

my $s = IO::Socket::SSL->new(
PeerHost => $h,

PeerPort => 443,

);

unless ($s) {

print "# error=$!, ssl_error=$SSL_ERROR\n";

print "not " unless $s;

}

print "ok ", ++$testnum, "\n";

}

when I run it I see:

1..2

# www.apple.com

ok 1

# www.microsoft.com

# error=, ssl_error=SSL connect attempt failed error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed

not ok 2

AFAICT this is the same issue as discussed in https://github.com/gisle/mozilla-ca/pull/5. I had Mozilla-CA-20141217 installed above.

Thu May 28 18:10:11 2015 GAAS [...] cpan.org - Correspondence added

... which lead me to https://github.com/kennethreitz/requests/issues/2455 and http://bugs.python.org/issue23476 and made me try this patch to IO-Socket-SSL:

diff --git a/lib/IO/Socket/SSL.pm b/lib/IO/Socket/SSL.pm

index 3118ea8..bae6391 100644

--- a/lib/IO/Socket/SSL.pm

+++ b/lib/IO/Socket/SSL.pm

@@ -2291,6 +2291,10 @@ WARN

return IO::Socket::SSL->error(

"Invalid default certificate authority locations")

}

+ Net::SSLeay::X509_STORE_set_flags(

+ Net::SSLeay::CTX_get_cert_store($ctx),

+ Net::SSLeay::X509_V_FLAG_TRUSTED_FIRST(),

+ ) if eval { Net::SSLeay::X509_V_FLAG_TRUSTED_FIRST() };

}

if ($arg_hash->{SSL_server}

which fixes this problem if you also fix your Net::SSLeay module to export this constant :-). I'm not so sure this is the right fix, but I hope you find this discovery helpful in some way.

Thu May 28 18:25:19 2015 GAAS [...] cpan.org - Correspondence added

On Thu May 28 18:10:11 2015, GAAS wrote:
Show quoted text

> which fixes this problem if you also fix your Net::SSLeay module to
> export this constant :-).

Show quoted text

#104760 contains the patch I applied Net-SSLeay.

Fri May 29 01:24:57 2015 Steffen_Ullrich [...] genua.de - Correspondence added

Show quoted text

> > which fixes this problem if you also fix your Net::SSLeay module to > export this > constant :-). I'm not so sure this is the right fix, but I hope you > find this > discovery helpful in some way.

Yes, this problem occurs a lot in the last time in all languages/tools/lib which use OpenSSL and a recent CA package from Mozilla, i.e. with the 1024 bit CA's removed. The problem is a long standing bug with OpenSSL and the handling of alternative trust path, more details at http://stackoverflow.com/questions/27804710/python-urllib2-ssl-error/27826829#27826829. I've reported the problem already 2012 to OpenSSL :( One workaround is to use X509_V_FLAG_TRUSTED_FIRST. But this option is only available since OpenSSL 1.02 which was only released 4 month ago (and introduced some other bugs now fixed with 1.02a :) Another workaround is to continue to use the 1024 bit certs, either by using the older Mozilla::CA 20130114, by using the old ca-bundle instead of the new from http://curl.haxx.se/docs/caextract.html or by using the distributions CA store which usually includes the older CA certificates too. Debian/Ubuntu patched LWP this way so you will not run into this problem there. The issue was fixed finally in 02/2015 but only in the newest OpenSSL branch (not 1.0.2x). So I don't expect a release in the near future. See https://rt.openssl.org/Ticket/Display.html?id=3637&user=guest&pass=guest. I think a good idea is to at least enable the flag by default if it is available (e.g. OpenSSL 1.0.2 and available from Net::SSLeay). Other then that there is unfortunately not much one can do except to educate users to keep using the 1024 bit CA's.

Fri May 29 01:24:58 2015 The RT System itself - Status changed from 'new' to 'open'

Fri May 29 02:03:44 2015 Steffen_Ullrich [...] genua.de - Correspondence added

Show quoted text

> I think a good idea is to at least enable the flag by default if it is > available (e.g. OpenSSL 1.0.2 and available from Net::SSLeay).

The flag will be enabled by default if available, starting with 2.015_005, see https://github.com/noxxi/p5-io-socket-ssl/commit/3ede5be99ebe4f0ae1d82c44caac8a3ff4a46451. Since I don't have OpenSSL 1.02 on my system - can you please check if the code works for you?

Sat May 30 03:33:08 2015 Steffen_Ullrich [...] genua.de - Correspondence added

Show quoted text

> .... Another > workaround is to continue to use the 1024 bit certs, either by using > the older Mozilla::CA 20130114, by using the old ca-bundle instead of > the new from http://curl.haxx.se/docs/caextract.html or by using the > distributions CA store which usually includes the older CA > certificates too. Debian/Ubuntu patched LWP this way so you will not > run into this problem there.

Hi Gisle, there is a pull request for LWP::Protocol::https which make it use the default trust store from IO::Socket::SSL. On most systems this means the systems CA store and only on systems which don't provide a useful store for OpenSSL it falls back to Mozilla::CA (like on Windows). In effect this would be the behavior Debian/Ubuntu already have know. This patch would lessen the problem for most systems. karenetheridge is willing to merge the patch but cannot ship are now version. But it looks like you could help, see https://github.com/libwww-perl/lwp-protocol-https/pull/15#issuecomment-106954948 Regards, Steffen

Tue Jun 02 15:05:30 2015 GAAS [...] cpan.org - Correspondence added

On Fri May 29 02:03:44 2015, SULLR wrote:
Show quoted text

>

> > I think a good idea is to at least enable the flag by default if it
> > is
> > available (e.g. OpenSSL 1.0.2 and available from Net::SSLeay).

>
> The flag will be enabled by default if available, starting with
> 2.015_005, see https://github.com/noxxi/p5-io-socket-
> ssl/commit/3ede5be99ebe4f0ae1d82c44caac8a3ff4a46451. Since I don't
> have OpenSSL 1.02 on my system - can you please check if the code
> works for you?

It doesn't work. The "defined &Net::SSLeay::X509_V_FLAG_TRUSTED_FIRST"-test is always false. I guess because the constant is not autoloaded yet.

I had to apply the following patch on top of yours to make it work here:

--- a/lib/IO/Socket/SSL.pm

+++ b/lib/IO/Socket/SSL.pm

@@ -232,7 +232,7 @@ if (!defined &Net::SSLeay::CTX_clear_options) {

# Try to work around problems with alternative trust path by default, RT#104759

my $DEFAULT_X509_STORE_flags = 0;

-if (defined &Net::SSLeay::X509_V_FLAG_TRUSTED_FIRST) {

+if (eval { Net::SSLeay::X509_V_FLAG_TRUSTED_FIRST(); }) {

$DEFAULT_X509_STORE_flags = Net::SSLeay::X509_V_FLAG_TRUSTED_FIRST();

}

Tue Jun 02 15:36:13 2015 Steffen_Ullrich [...] genua.de - Correspondence added

Show quoted text

> > It doesn't work. The "defined > &Net::SSLeay::X509_V_FLAG_TRUSTED_FIRST"-test is > always false. I guess because the constant is not autoloaded yet.

2.015_006 should do it then hopefully: my $DEFAULT_X509_STORE_flags = 0; -if (defined &Net::SSLeay::X509_V_FLAG_TRUSTED_FIRST) { - $DEFAULT_X509_STORE_flags = Net::SSLeay::X509_V_FLAG_TRUSTED_FIRST(); -} +eval { $DEFAULT_X509_STORE_flags |= Net::SSLeay::X509_V_FLAG_TRUSTED_FIRST() };

Tue Jun 02 16:21:47 2015 GAAS [...] cpan.org - Correspondence added

I've now verified that this patch works for me.

Tue Jun 02 16:41:38 2015 Steffen_Ullrich [...] genua.de - Correspondence added

Am Di 02. Jun 2015, 16:21:47, GAAS schrieb: Show quoted text

> I've now verified that this patch works for me.

Thanks for testing. This is now released as 2.016.

Tue Jun 02 16:41:46 2015 Steffen_Ullrich [...] genua.de - Status changed from 'open' to 'resolved'