Skip Menu |

This queue is for tickets about the Net-SSLeay CPAN distribution.

Report information
The Basics
Id: 116795
Status: resolved
Priority: 0/
Queue: Net-SSLeay

People
Owner: Nobody in particular
Requestors: Steffen_Ullrich [...] genua.de
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in:
  • 1.75
  • 1.76
  • 1.77
Fixed in: (no value)



Subject: OCSP broken in 1.75+ - patch included
Hi, Due to code changes OCSP got broken in 1.75. Unfortunately the existing test showed no error because due to changed certificates for the external sites the test simply was skipped automatically. Attached is a patch which makes OCSP work again. This also updates the patch to match the new certificates and additionally makes sure that the OCSP test will not skip subtests when the certificates change again, at least not if ENV RELEASE_TESTING is set. Regards, Steffen
Subject: patch.txt
Index: SSLeay.xs =================================================================== --- SSLeay.xs (revision 477) +++ SSLeay.xs (working copy) @@ -6011,7 +6011,7 @@ X509 *issuer; X509 *last = sk_X509_value(chain,sk_X509_num(chain)-1); if ( (issuer = find_issuer(last,store,chain))) { - OCSP_basic_add1_cert(bsr, X509_dup(issuer)); + OCSP_basic_add1_cert(bsr, issuer); TRACE(1,"run OCSP_basic_verify with issuer for last chain element"); RETVAL = OCSP_basic_verify(bsr, NULL, store, flags); } @@ -6058,11 +6058,8 @@ goto end; } int first = OCSP_resp_find(bsr, certid, -1); /* Find the first matching */ - if (first >= 0) - { - sir = OCSP_resp_get0(bsr,first); - break; - } + if (first >= 0) + sir = OCSP_resp_get0(bsr,first); } int status, revocationReason; @@ -6073,7 +6070,8 @@ status = OCSP_single_get0_status(sir, &revocationReason, &revocationTime, &thisupdate, &nextupdate); #else status = sir->certStatus->type; - revocationTime = sir->certStatus->value.revoked->revocationTime; + if (status == V_OCSP_CERTSTATUS_REVOKED) + revocationTime = sir->certStatus->value.revoked->revocationTime; thisupdate = sir->thisUpdate; nextupdate = sir->nextUpdate; #endif Index: t/external/ocsp.t =================================================================== --- t/external/ocsp.t (revision 477) +++ t/external/ocsp.t (working copy) @@ -16,7 +16,7 @@ # this should give us OCSP stapling host => 'www.live.com', port => 443, - fingerprint => '10c56ee9e2acaf2e77caeb7072bf6522dd7422b8', + fingerprint => '0e37dc9b320d2526e93e360a26c824b202d1f3af', ocsp_staple => 1, expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), }, @@ -24,7 +24,7 @@ # no OCSP stapling yet host => 'www.google.com', port => 443, - fingerprint => '007a5ab302f14446e2ea24d3a829de22ba1bf950', + fingerprint => '89380c438a076d9d5fac228a8f680ff452487f30', expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), }, { @@ -31,7 +31,7 @@ # this is revoked host => 'revoked.grc.com', port => 443, - fingerprint => '34703c40093461ad3ce087e161c7b7f42abe770c', + fingerprint => '310665f4c8e78db761c764e798dca66047341264', expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_REVOKED(), }, ); @@ -50,6 +50,12 @@ TEST: +sub skip_unless_release { + die "this test should not be skipped for release - might need to fix test" + if $ENV{RELEASE_TESTING}; + goto &skip; +} + for my $test (@tests) { my $cleanup = __cleanup__->new; SKIP: { @@ -61,7 +67,7 @@ PeerPort => $test->{port}, Timeout => $timeout, ); - skip "TCP connect to $test->{host}:$test->{port} failed: $!",1 + skip_unless_release "TCP connect to $test->{host}:$test->{port} failed: $!",1 if !$cl; diag("tcp connect to $test->{host}:$test->{port} ok"); @@ -103,7 +109,7 @@ select(undef,$vec,undef,$to); } } - skip "SSL_connect with $test->{host}:$test->{port} failed: ". + skip_unless_release "SSL_connect with $test->{host}:$test->{port} failed: ". Net::SSLeay::print_errs(''),1 if $rv<=0; diag("SSL_connect ok"); @@ -113,8 +119,8 @@ $cleanup->add(sub { Net::SSLeay::X509_free($leaf_cert) }) if $leaf_cert; my $fp = $leaf_cert && unpack("H*",Net::SSLeay::X509_digest($leaf_cert,$sha1)); - skip "could not get fingerprint",1 if !$fp; - skip "bad fingerprint $fp for $test->{host}:$test->{port}",1 + skip_unless_release "could not get fingerprint",1 if !$fp; + skip_unless_release "bad fingerprint $fp for $test->{host}:$test->{port}",1 if $fp ne $test->{fingerprint}; diag("fingerprint matches");
Subject: Re: [rt.cpan.org #116795] OCSP broken in 1.75+ - patch included
Date: Wed, 10 Aug 2016 09:40:55 +1000
To: bug-Net-SSLeay [...] rt.cpan.org
From: Mike McCauley <mikem [...] airspayce.com>
Thanks Steffen, I will apply this soon, but Im not sure this code in ocsp.t is helpful: sub skip_unless_release { die "this test should not be skipped for release - might need to fix test" if $ENV{RELEASE_TESTING}; goto &skip; } If you enable RELEASE_TESTING the test fail with: t/external/ocsp.t ...................... # tcp connect to www.live.com:443 ok # got stapled OCSP response # SSL_connect ok this test should not be skipped for release - might need to fix test at t/external/ocsp.t line 54. # Looks like your test exited with 255 before it could output anything. t/external/ocsp.t ...................... Dubious, test returned 255 (wstat 65280, 0xff00) Failed 3/3 subtests I dont think die ing is the right response. Perhaps you meant return if $ENV{RELEASE_TESTING}; Cheers. On Tuesday, August 09, 2016 10:29:04 AM you wrote: Show quoted text
> Tue Aug 09 10:29:02 2016: Request 116795 was acted upon. > Transaction: Ticket created by SULLR > Queue: Net-SSLeay > Subject: OCSP broken in 1.75+ - patch included > Broken in: 1.75, 1.76, 1.77 > Severity: (no value) > Owner: Nobody > Requestors: Steffen_Ullrich@genua.de > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=116795 > > > > Hi, > > Due to code changes OCSP got broken in 1.75. Unfortunately the existing test > showed no error because due to changed certificates for the external sites > the test simply was skipped automatically. > > Attached is a patch which makes OCSP work again. This also updates the patch > to match the new certificates and additionally makes sure that the OCSP > test will not skip subtests when the certificates change again, at least > not if ENV RELEASE_TESTING is set. > > Regards, > Steffen
-- Mike McCauley VK4AMM mikem@airspayce.com Airspayce Pty Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.airspayce.com Phone +61 7 5598-7474
Subject: Re: [rt.cpan.org #116795] OCSP broken in 1.75+ - patch included
Date: Wed, 10 Aug 2016 07:44:12 +0200
To: Mike McCauley via RT <bug-Net-SSLeay [...] rt.cpan.org>
From: Steffen Ullrich <Steffen_Ullrich [...] genua.de>
Show quoted text
> > I will apply this soon, but Im not sure this code in ocsp.t is helpful: > > sub skip_unless_release { > die "this test should not be skipped for release - might need to fix test" > ... > I dont think die ing is the right response. Perhaps you meant > > return > if $ENV{RELEASE_TESTING};
The problem I've tried to solve was that the OCSP test automatically got skipped because the fingerprints of the certificates did not match anymore and thus SSL interception was assumed (which makes OCSP tests not possible). The intention of the die() was that the test should fail badly in this case on RELEASE_TESTING so that one adjusts the fingerprints if needed. With the return instead of the die it will simply assume that the fingerprints are valid and continue. This might either result in that the test passes (if only the certificates got changed but the OCSP behavior stays) or that the tests fail (if there is SSL interception or the OCSP behavior changed). I would rather see that the developer is forced to adapt the test to the new certifcates, because otherwise the test gets automatically skipped for all others not doing release tests because of the certificate mismatch. That's why I've opted for the die(). Regards, Steffen
Subject: Re: [rt.cpan.org #116795] OCSP broken in 1.75+ - patch included
Date: Thu, 11 Aug 2016 09:23:03 +1000
To: bug-Net-SSLeay [...] rt.cpan.org
From: Mike McCauley <mikem [...] airspayce.com>
Hi Steffen, On Wednesday, August 10, 2016 01:44:34 AM you wrote: Show quoted text
> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=116795 > >
> > I will apply this soon, but Im not sure this code in ocsp.t is helpful: > > > > sub skip_unless_release { > > > > die "this test should not be skipped for release - might need to fix > > test" > > > > ... > > I dont think die ing is the right response. Perhaps you meant > > > > return > > > > if $ENV{RELEASE_TESTING};
> > The problem I've tried to solve was that the OCSP test automatically got > skipped because the fingerprints of the certificates did not match anymore > and thus SSL interception was assumed (which makes OCSP tests not possible). > > The intention of the die() was that the test should fail badly in this case > on RELEASE_TESTING so that one adjusts the fingerprints if needed. With the > return instead of the die it will simply assume that the fingerprints are > valid and continue. This might either result in that the test passes (if > only the certificates got changed but the OCSP behavior stays) or that the > tests fail (if there is SSL interception or the OCSP behavior changed). > > I would rather see that the developer is forced to adapt the test to the new > certifcates, because otherwise the test gets automatically skipped for all > others not doing release tests because of the certificate mismatch. That's > why I've opted for the die().
Im not sure I understand. If I 'make test' with RELEASE_TESTING=1 then it dies with no further info, so how does one know whether the keys are correct and what to change them to? When you say 'the developer is forced', I guess you mean me? What exactly am I forced to do, and when? Cheers. Show quoted text
> > Regards, > Steffen
-- Mike McCauley VK4AMM mikem@airspayce.com Airspayce Pty Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.airspayce.com Phone +61 7 5598-7474
Show quoted text
> When you say 'the developer is forced', I guess you mean me? > > What exactly am I forced to do, and when?
I've changed the test now that it does not die() anymore. Instead it will give information about the expected and the new fingerprint if the fingerprint match fails. I've also added an additional test only if RELEASE_TESTING is set. This test will fail if subtests where skipped because the fingerprint of the certificate mismatched. Since the diagnostics show the host, the old and the new fingerprint it should be possible for the developer (i.e. you) to quickly change the test so that it uses the new fingerprint of the certificate. And since this last test only runs if RELEASE_TESTING is set it will not affect normal testing. Regards, Steffen
forgot the new patch
Subject: patch.txt
Index: SSLeay.xs =================================================================== --- SSLeay.xs (revision 477) +++ SSLeay.xs (working copy) @@ -6011,7 +6011,7 @@ X509 *issuer; X509 *last = sk_X509_value(chain,sk_X509_num(chain)-1); if ( (issuer = find_issuer(last,store,chain))) { - OCSP_basic_add1_cert(bsr, X509_dup(issuer)); + OCSP_basic_add1_cert(bsr, issuer); TRACE(1,"run OCSP_basic_verify with issuer for last chain element"); RETVAL = OCSP_basic_verify(bsr, NULL, store, flags); } @@ -6058,11 +6058,8 @@ goto end; } int first = OCSP_resp_find(bsr, certid, -1); /* Find the first matching */ - if (first >= 0) - { - sir = OCSP_resp_get0(bsr,first); - break; - } + if (first >= 0) + sir = OCSP_resp_get0(bsr,first); } int status, revocationReason; @@ -6073,7 +6070,8 @@ status = OCSP_single_get0_status(sir, &revocationReason, &revocationTime, &thisupdate, &nextupdate); #else status = sir->certStatus->type; - revocationTime = sir->certStatus->value.revoked->revocationTime; + if (status == V_OCSP_CERTSTATUS_REVOKED) + revocationTime = sir->certStatus->value.revoked->revocationTime; thisupdate = sir->thisUpdate; nextupdate = sir->nextUpdate; #endif Index: t/external/ocsp.t =================================================================== --- t/external/ocsp.t (revision 477) +++ t/external/ocsp.t (working copy) @@ -16,7 +16,7 @@ # this should give us OCSP stapling host => 'www.live.com', port => 443, - fingerprint => '10c56ee9e2acaf2e77caeb7072bf6522dd7422b8', + fingerprint => '0e37dc9b320d2526e93e360a26c824b202d1f3af', ocsp_staple => 1, expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), }, @@ -24,7 +24,7 @@ # no OCSP stapling yet host => 'www.google.com', port => 443, - fingerprint => '007a5ab302f14446e2ea24d3a829de22ba1bf950', + fingerprint => '89380c438a076d9d5fac228a8f680ff452487f30', expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), }, { @@ -31,12 +31,13 @@ # this is revoked host => 'revoked.grc.com', port => 443, - fingerprint => '34703c40093461ad3ce087e161c7b7f42abe770c', + fingerprint => '310665f4c8e78db761c764e798dca66047341264', expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_REVOKED(), }, ); -plan tests => 0+@tests; +my $release_tests = $ENV{RELEASE_TESTING} && 1; +plan tests => $release_tests + @tests; my $timeout = 10; # used to TCP connect and SSL connect @@ -50,6 +51,12 @@ TEST: +my $skipped = 0; +sub skip_unless_release { + $skipped++ if $release_tests; + goto &skip; +} + for my $test (@tests) { my $cleanup = __cleanup__->new; SKIP: { @@ -114,8 +121,9 @@ my $fp = $leaf_cert && unpack("H*",Net::SSLeay::X509_digest($leaf_cert,$sha1)); skip "could not get fingerprint",1 if !$fp; - skip "bad fingerprint $fp for $test->{host}:$test->{port}",1 - if $fp ne $test->{fingerprint}; + skip_unless_release( + "bad fingerprint $fp for $test->{host}:$test->{port} - expected $test->{fingerprint}, got $fp", + 1) if $fp ne $test->{fingerprint}; diag("fingerprint matches"); if ( $test->{ocsp_staple} && ! $stapled_response ) { @@ -225,6 +233,8 @@ } } +is($skipped,0,"skipped critical tests for release") if $release_tests; + { # cleanup stuff when going out of scope package __cleanup__;
Subject: Re: [rt.cpan.org #116795] OCSP broken in 1.75+ - patch included
Date: Thu, 11 Aug 2016 21:02:58 +1000
To: bug-Net-SSLeay [...] rt.cpan.org
From: Mike McCauley <mikem [...] airspayce.com>
Hi Steffen, thanks for the update, it looks better, but now when I run: RELEASE_TESTING=1 make test I get: ... t/external/ocsp.t ...................... # tcp connect to www.live.com:443 ok # got stapled OCSP response # SSL_connect ok t/external/ocsp.t ...................... 1/4 # tcp connect to www.google.com:443 ok # got no stapled OCSP response # SSL_connect ok t/external/ocsp.t ...................... 2/4 # tcp connect to revoked.grc.com:443 ok # got stapled OCSP response # SSL_connect ok # fingerprint matches # status=1 as expected: nextUpd=Fri Aug 12 12:37:11 2016 # status=1 as expected: nextUpd=Mon Aug 15 10:58:41 2016 # status=0 as expected: nextUpd=Mon Aug 15 10:58:42 2016 t/external/ocsp.t ...................... 3/4 # Failed test 'skipped critical tests for release' # at t/external/ocsp.t line 236. # got: '2' # expected: '0' # Looks like you failed 1 test of 4. t/external/ocsp.t ...................... Dubious, test returned 1 (wstat 256, 0x100) That doesnt look right? Cheers. On Thursday, August 11, 2016 06:02:50 AM Steffen Ullrich via RT wrote: Show quoted text
> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=116795 > >
> > When you say 'the developer is forced', I guess you mean me? > > > > What exactly am I forced to do, and when?
> > I've changed the test now that it does not die() anymore. > Instead it will give information about the expected and the new fingerprint > if the fingerprint match fails. I've also added an additional test only if > RELEASE_TESTING is set. This test will fail if subtests where skipped > because the fingerprint of the certificate mismatched. Since the > diagnostics show the host, the old and the new fingerprint it should be > possible for the developer (i.e. you) to quickly change the test so that it > uses the new fingerprint of the certificate. And since this last test only > runs if RELEASE_TESTING is set it will not affect normal testing. > > Regards, > Steffen
-- Mike McCauley VK4AMM mikem@airspayce.com Airspayce Pty Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.airspayce.com Phone +61 7 5598-7474
Am Do 11. Aug 2016, 07:04:14, mikem@airspayce.com schrieb: Show quoted text
> Hi Steffen, > > thanks for the update, it looks better, but now when I run: > > RELEASE_TESTING=1 make test
I see, the output from 'make test' does not include the reasons why tests got skipped. I've tested before with perl -Mblib and there it showed up. I've now adapted the test so that it uses diag() and it looks like this shows up in 'make test'. Apart from that it looks like that both live.com and google.com use different certificates in different parts of the world (i.e. based on target IP address) which made the test work for me but not for you :(. I've now switched to different hosts although it is hard to find a host which supports OCSP stapling. Attached is a new patch, this time for the test only. Regards, Steffen
Subject: patch.txt
Index: SSLeay.xs =================================================================== --- SSLeay.xs (revision 477) +++ SSLeay.xs (working copy) @@ -6011,7 +6011,7 @@ X509 *issuer; X509 *last = sk_X509_value(chain,sk_X509_num(chain)-1); if ( (issuer = find_issuer(last,store,chain))) { - OCSP_basic_add1_cert(bsr, X509_dup(issuer)); + OCSP_basic_add1_cert(bsr, issuer); TRACE(1,"run OCSP_basic_verify with issuer for last chain element"); RETVAL = OCSP_basic_verify(bsr, NULL, store, flags); } @@ -6058,11 +6058,8 @@ goto end; } int first = OCSP_resp_find(bsr, certid, -1); /* Find the first matching */ - if (first >= 0) - { - sir = OCSP_resp_get0(bsr,first); - break; - } + if (first >= 0) + sir = OCSP_resp_get0(bsr,first); } int status, revocationReason; @@ -6073,7 +6070,8 @@ status = OCSP_single_get0_status(sir, &revocationReason, &revocationTime, &thisupdate, &nextupdate); #else status = sir->certStatus->type; - revocationTime = sir->certStatus->value.revoked->revocationTime; + if (status == V_OCSP_CERTSTATUS_REVOKED) + revocationTime = sir->certStatus->value.revoked->revocationTime; thisupdate = sir->thisUpdate; nextupdate = sir->nextUpdate; #endif Index: t/external/ocsp.t =================================================================== --- t/external/ocsp.t (revision 477) +++ t/external/ocsp.t (working copy) @@ -14,17 +14,17 @@ my @tests = ( { # this should give us OCSP stapling - host => 'www.live.com', + host => 'www.microsoft.com', port => 443, - fingerprint => '10c56ee9e2acaf2e77caeb7072bf6522dd7422b8', + fingerprint => '5f0b37e633840ca02468552ea3b1197e5e118f7b', ocsp_staple => 1, expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), }, { - # no OCSP stapling yet - host => 'www.google.com', + # no OCSP stapling + host => 'www.spiegel.de', port => 443, - fingerprint => '007a5ab302f14446e2ea24d3a829de22ba1bf950', + fingerprint => 'ad737048455485d8c817b7d0f7403553a7b9f65b', expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), }, { @@ -31,12 +31,13 @@ # this is revoked host => 'revoked.grc.com', port => 443, - fingerprint => '34703c40093461ad3ce087e161c7b7f42abe770c', + fingerprint => '310665f4c8e78db761c764e798dca66047341264', expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_REVOKED(), }, ); -plan tests => 0+@tests; +my $release_tests = $ENV{RELEASE_TESTING} ? 1:0; +plan tests => $release_tests + @tests; my $timeout = 10; # used to TCP connect and SSL connect @@ -50,6 +51,7 @@ TEST: +my @fp_mismatch; for my $test (@tests) { my $cleanup = __cleanup__->new; SKIP: { @@ -114,8 +116,11 @@ my $fp = $leaf_cert && unpack("H*",Net::SSLeay::X509_digest($leaf_cert,$sha1)); skip "could not get fingerprint",1 if !$fp; - skip "bad fingerprint $fp for $test->{host}:$test->{port}",1 - if $fp ne $test->{fingerprint}; + if ($fp ne $test->{fingerprint}) { + push @fp_mismatch, [ $fp,$test ]; + skip("bad fingerprint for $test->{host}:$test->{port} -". + " expected $test->{fingerprint}, got $fp",1) + } diag("fingerprint matches"); if ( $test->{ocsp_staple} && ! $stapled_response ) { @@ -225,6 +230,19 @@ } } +if ($release_tests) { + if (!@fp_mismatch) { + pass("all fingerprints matched"); + } else { + for(@fp_mismatch) { + my ($fp,$test) = @$_; + diag("fingerprint mismatch for $test->{host}:$test->{port} -". + " expected $test->{fingerprint}, got $fp") + } + fail("some fingerprints did not matched - please adjust test"); + } +} + { # cleanup stuff when going out of scope package __cleanup__;
Subject: Re: [rt.cpan.org #116795] OCSP broken in 1.75+ - patch included
Date: Fri, 12 Aug 2016 10:59:35 +1000
To: bug-Net-SSLeay [...] rt.cpan.org
From: Mike McCauley <mikem [...] airspayce.com>
Hi Steffen, thanks, that works well with my builtin 1.0.1k-fips , libressl-2.4.1 and others, but with openssl-1.1.0, openssl-1.0.2c, openssl-1.0.2a, openssl-1.0.2, openssl-1.0.1c and other (without RELEASE_TESTING) I get: t/external/ocsp.t ...................... # tcp connect to www.microsoft.com:443 ok # got stapled OCSP response # SSL_connect ok # fingerprint matches t/external/ocsp.t ...................... 1/3 # Failed test 'failed to get OCSP_CERTIDs for cert /C=US/O=Symantec Corporation/OU=Symantec Trust Network/CN=Symantec Class 3 Secure Server CA - G4: cannot find issuer to certificate at t/external/ocsp.t line 141. # ' # at t/external/ocsp.t line 142. Label not found for "next TEST" at t/external/ocsp.t line 143. # Looks like your test exited with 2 just after 1. t/external/ocsp.t ...................... Dubious, test returned 2 (wstat 512, 0x200) That Label not found for "next TEST" is weird? perl 5, version 20, subversion 1 (v5.20.1) built for i586-linux-thread-multi Cheers. On Thursday, August 11, 2016 08:01:51 AM you wrote: Show quoted text
> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=116795 > > > Am Do 11. Aug 2016, 07:04:14, mikem@airspayce.com schrieb:
> > Hi Steffen, > > > > thanks for the update, it looks better, but now when I run: > > > > RELEASE_TESTING=1 make test
> > I see, the output from 'make test' does not include the reasons why tests > got skipped. I've tested before with perl -Mblib and there it showed up. > I've now adapted the test so that it uses diag() and it looks like this > shows up in 'make test'. > > Apart from that it looks like that both live.com and google.com use > different certificates in different parts of the world (i.e. based on > target IP address) which made the test work for me but not for you :(. I've > now switched to different hosts although it is hard to find a host which > supports OCSP stapling. > > Attached is a new patch, this time for the test only. > > Regards, > Steffen
-- Mike McCauley VK4AMM mikem@airspayce.com Airspayce Pty Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.airspayce.com Phone +61 7 5598-7474
Show quoted text
> thanks, that works well with my builtin 1.0.1k-fips , libressl-2.4.1 > and > others, but with > openssl-1.1.0, openssl-1.0.2c, openssl-1.0.2a, openssl-1.0.2, openssl- > 1.0.1c
I've tried with 1.0.2c, 1.1.0, 1.0.0d, 1.0.2g-fips (ubuntu) so there is at least some overlap in the versions. Since it fails in OCSP_CERTIDs with "cannot find issuer to certificate" it might be that you have different CA storage setup with the different OpenSSL versions and that some of the CA storage don't contain the necessary root certificate. This might happen if you have sometimes Mozilla::CA installed and sometimes not, in which case it will use the default store. On the other hand it has certificate validation enabled when connecting, i.e. in case of missing root-CA it should have been skipped this test because the connection failed. Very strange indeed. Show quoted text
> Label not found for "next TEST"
That happened because I've inadvertently placed a variable declaration between the label and the for loop and thus the label got not associated with the loop anymore. This is fixed within the attached patch.
Subject: patch.txt
Index: SSLeay.xs =================================================================== --- SSLeay.xs (revision 477) +++ SSLeay.xs (working copy) @@ -5916,7 +5916,7 @@ if (X509_check_issued(cert,cert) == X509_V_OK) croak("no OCSP request for self-signed certificate"); if (!(issuer = find_issuer(cert,store,chain))) - croak("cannot find issuer to certificate"); + croak("cannot find issuer certificate"); if (!(id = OCSP_cert_to_id(EVP_sha1(),cert,issuer))) croak("out of memory for generating OCSO certid"); if (!(len = i2d_OCSP_CERTID(id,NULL))) @@ -6011,7 +6011,7 @@ X509 *issuer; X509 *last = sk_X509_value(chain,sk_X509_num(chain)-1); if ( (issuer = find_issuer(last,store,chain))) { - OCSP_basic_add1_cert(bsr, X509_dup(issuer)); + OCSP_basic_add1_cert(bsr, issuer); TRACE(1,"run OCSP_basic_verify with issuer for last chain element"); RETVAL = OCSP_basic_verify(bsr, NULL, store, flags); } @@ -6058,11 +6058,8 @@ goto end; } int first = OCSP_resp_find(bsr, certid, -1); /* Find the first matching */ - if (first >= 0) - { - sir = OCSP_resp_get0(bsr,first); - break; - } + if (first >= 0) + sir = OCSP_resp_get0(bsr,first); } int status, revocationReason; @@ -6073,7 +6070,8 @@ status = OCSP_single_get0_status(sir, &revocationReason, &revocationTime, &thisupdate, &nextupdate); #else status = sir->certStatus->type; - revocationTime = sir->certStatus->value.revoked->revocationTime; + if (status == V_OCSP_CERTSTATUS_REVOKED) + revocationTime = sir->certStatus->value.revoked->revocationTime; thisupdate = sir->thisUpdate; nextupdate = sir->nextUpdate; #endif Index: t/external/ocsp.t =================================================================== --- t/external/ocsp.t (revision 477) +++ t/external/ocsp.t (working copy) @@ -14,17 +14,17 @@ my @tests = ( { # this should give us OCSP stapling - host => 'www.live.com', + host => 'www.microsoft.com', port => 443, - fingerprint => '10c56ee9e2acaf2e77caeb7072bf6522dd7422b8', + fingerprint => '5f0b37e633840ca02468552ea3b1197e5e118f7b', ocsp_staple => 1, expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), }, { - # no OCSP stapling yet - host => 'www.google.com', + # no OCSP stapling + host => 'www.spiegel.de', port => 443, - fingerprint => '007a5ab302f14446e2ea24d3a829de22ba1bf950', + fingerprint => 'ad737048455485d8c817b7d0f7403553a7b9f65b', expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), }, { @@ -31,12 +31,13 @@ # this is revoked host => 'revoked.grc.com', port => 443, - fingerprint => '34703c40093461ad3ce087e161c7b7f42abe770c', + fingerprint => '310665f4c8e78db761c764e798dca66047341264', expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_REVOKED(), }, ); -plan tests => 0+@tests; +my $release_tests = $ENV{RELEASE_TESTING} ? 1:0; +plan tests => $release_tests + @tests; my $timeout = 10; # used to TCP connect and SSL connect @@ -48,8 +49,9 @@ Net::SSLeay::SSLeay_add_ssl_algorithms(); my $sha1 = Net::SSLeay::EVP_get_digestbyname('sha1'); + +my @fp_mismatch; TEST: - for my $test (@tests) { my $cleanup = __cleanup__->new; SKIP: { @@ -114,8 +116,11 @@ my $fp = $leaf_cert && unpack("H*",Net::SSLeay::X509_digest($leaf_cert,$sha1)); skip "could not get fingerprint",1 if !$fp; - skip "bad fingerprint $fp for $test->{host}:$test->{port}",1 - if $fp ne $test->{fingerprint}; + if ($fp ne $test->{fingerprint}) { + push @fp_mismatch, [ $fp,$test ]; + skip("bad fingerprint for $test->{host}:$test->{port} -". + " expected $test->{fingerprint}, got $fp",1) + } diag("fingerprint matches"); if ( $test->{ocsp_staple} && ! $stapled_response ) { @@ -225,6 +230,19 @@ } } +if ($release_tests) { + if (!@fp_mismatch) { + pass("all fingerprints matched"); + } else { + for(@fp_mismatch) { + my ($fp,$test) = @$_; + diag("fingerprint mismatch for $test->{host}:$test->{port} -". + " expected $test->{fingerprint}, got $fp") + } + fail("some fingerprints did not matched - please adjust test"); + } +} + { # cleanup stuff when going out of scope package __cleanup__;
Subject: Re: [rt.cpan.org #116795] OCSP broken in 1.75+ - patch included
Date: Fri, 12 Aug 2016 16:34:38 +1000
To: bug-Net-SSLeay [...] rt.cpan.org
From: Mike McCauley <mikem [...] airspayce.com>
Hi Steffen, Thanks. Works fine with libressl-2.4.1 and my builtin OpenSSL 1.0.1k-fips With openssl-1.1.0, openssl-1.0.2c etc I get: t/external/ocsp.t ...................... # tcp connect to www.microsoft.com:443 ok # got stapled OCSP response # SSL_connect ok # fingerprint matches t/external/ocsp.t ...................... 1/3 # Failed test 'failed to get OCSP_CERTIDs for cert /C=US/O=Symantec Corporation/OU=Symantec Trust Network/CN=Symantec Class 3 Secure Server CA - G4: cannot find issuer certificate at t/external/ocsp.t line 141. # ' # at t/external/ocsp.t line 142. # tcp connect to www.spiegel.de:443 ok # got no stapled OCSP response # SSL_connect ok # fingerprint matches # no OCSP URI for cert /C=US/O=GeoTrust Inc./CN=GeoTrust Global CA t/external/ocsp.t ...................... 2/3 # Failed test 'cannot verify response: ' # at t/external/ocsp.t line 224. # tcp connect to revoked.grc.com:443 ok # got stapled OCSP response # SSL_connect ok # fingerprint matches t/external/ocsp.t ...................... 3/3 # Failed test 'failed to get OCSP_CERTIDs for cert /C=BE/O=GlobalSign nv- sa/CN=GlobalSign Domain Validation CA - G2: cannot find issuer certificate at t/external/ocsp.t line 141. # ' # at t/external/ocsp.t line 142. # Looks like you failed 3 tests of 3. t/external/ocsp.t ...................... Dubious, test returned 3 (wstat 768, 0x300) Cheers. On Friday, August 12, 2016 02:02:06 AM you wrote: Show quoted text
> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=116795 > >
> > thanks, that works well with my builtin 1.0.1k-fips , libressl-2.4.1 > > and > > others, but with > > openssl-1.1.0, openssl-1.0.2c, openssl-1.0.2a, openssl-1.0.2, openssl- > > 1.0.1c
> > I've tried with 1.0.2c, 1.1.0, 1.0.0d, 1.0.2g-fips (ubuntu) so there is at > least some overlap in the versions. Since it fails in OCSP_CERTIDs with > "cannot find issuer to certificate" it might be that you have different CA > storage setup with the different OpenSSL versions and that some of the CA > storage don't contain the necessary root certificate. This might happen if > you have sometimes Mozilla::CA installed and sometimes not, in which case > it will use the default store. > > On the other hand it has certificate validation enabled when connecting, > i.e. in case of missing root-CA it should have been skipped this test > because the connection failed. Very strange indeed. >
> > Label not found for "next TEST"
> > That happened because I've inadvertently placed a variable declaration > between the label and the for loop and thus the label got not associated > with the loop anymore. This is fixed within the attached patch.
-- Mike McCauley VK4AMM mikem@airspayce.com Airspayce Pty Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.airspayce.com Phone +61 7 5598-7474
Show quoted text
> With openssl-1.1.0, openssl-1.0.2c etc I get: > ... > # Failed test 'failed to get OCSP_CERTIDs for cert /C=US/O=Symantec > Corporation/OU=Symantec Trust Network/CN=Symantec Class 3 Secure > Server CA - > G4: cannot find issuer certificate at t/external/ocsp.t line 141.
This is hopefully solved now. Inside the test I've first created the context and the SSL object and then added the features to the context I need. It looks like that some features in the SSL object are taken from the linked context while other features are copied from the context on creation of the SSL object and further changes to the context will not affect the SSL object. One of these copied features is the verification mode. Due to the order of creation of SSL object and context the SSL object ended up with a mode of VERIFY_NONE even if the context was set to VERIFY_PEER. And this caused the SSL_connect to succeed even though the certificate chain could not be verified against trusted CA. And because of the missing root CA the OCSP_CERTID failed to get the issue certificate. This is now fixed by first creating and fully setting up the context before creating the SSL object. This will probably mean that with your openssl-1.1.0 etc setups the test will be skipped because the SSL connections fail with "certificate verify failed". To fix this you might need to install Mozilla::CA in your test environment since it looks like that it cannot find the needed root CA.
Subject: patch.txt
Index: SSLeay.xs =================================================================== --- SSLeay.xs (revision 477) +++ SSLeay.xs (working copy) @@ -5916,7 +5916,7 @@ if (X509_check_issued(cert,cert) == X509_V_OK) croak("no OCSP request for self-signed certificate"); if (!(issuer = find_issuer(cert,store,chain))) - croak("cannot find issuer to certificate"); + croak("cannot find issuer certificate"); if (!(id = OCSP_cert_to_id(EVP_sha1(),cert,issuer))) croak("out of memory for generating OCSO certid"); if (!(len = i2d_OCSP_CERTID(id,NULL))) @@ -6011,7 +6011,7 @@ X509 *issuer; X509 *last = sk_X509_value(chain,sk_X509_num(chain)-1); if ( (issuer = find_issuer(last,store,chain))) { - OCSP_basic_add1_cert(bsr, X509_dup(issuer)); + OCSP_basic_add1_cert(bsr, issuer); TRACE(1,"run OCSP_basic_verify with issuer for last chain element"); RETVAL = OCSP_basic_verify(bsr, NULL, store, flags); } @@ -6058,11 +6058,8 @@ goto end; } int first = OCSP_resp_find(bsr, certid, -1); /* Find the first matching */ - if (first >= 0) - { - sir = OCSP_resp_get0(bsr,first); - break; - } + if (first >= 0) + sir = OCSP_resp_get0(bsr,first); } int status, revocationReason; @@ -6073,7 +6070,8 @@ status = OCSP_single_get0_status(sir, &revocationReason, &revocationTime, &thisupdate, &nextupdate); #else status = sir->certStatus->type; - revocationTime = sir->certStatus->value.revoked->revocationTime; + if (status == V_OCSP_CERTSTATUS_REVOKED) + revocationTime = sir->certStatus->value.revoked->revocationTime; thisupdate = sir->thisUpdate; nextupdate = sir->nextUpdate; #endif Index: t/external/ocsp.t =================================================================== --- t/external/ocsp.t (revision 477) +++ t/external/ocsp.t (working copy) @@ -14,17 +14,17 @@ my @tests = ( { # this should give us OCSP stapling - host => 'www.live.com', + host => 'www.microsoft.com', port => 443, - fingerprint => '10c56ee9e2acaf2e77caeb7072bf6522dd7422b8', + fingerprint => '5f0b37e633840ca02468552ea3b1197e5e118f7b', ocsp_staple => 1, expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), }, { - # no OCSP stapling yet - host => 'www.google.com', + # no OCSP stapling + host => 'www.heise.de', port => 443, - fingerprint => '007a5ab302f14446e2ea24d3a829de22ba1bf950', + fingerprint => '36a7d7bfc59db65c040bccd291ae563d9ef7bafc', expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), }, { @@ -31,12 +31,13 @@ # this is revoked host => 'revoked.grc.com', port => 443, - fingerprint => '34703c40093461ad3ce087e161c7b7f42abe770c', + fingerprint => '310665f4c8e78db761c764e798dca66047341264', expect_status => Net::SSLeay::V_OCSP_CERTSTATUS_REVOKED(), }, ); -plan tests => 0+@tests; +my $release_tests = $ENV{RELEASE_TESTING} ? 1:0; +plan tests => $release_tests + @tests; my $timeout = 10; # used to TCP connect and SSL connect @@ -48,8 +49,9 @@ Net::SSLeay::SSLeay_add_ssl_algorithms(); my $sha1 = Net::SSLeay::EVP_get_digestbyname('sha1'); + +my @fp_mismatch; TEST: - for my $test (@tests) { my $cleanup = __cleanup__->new; SKIP: { @@ -66,7 +68,6 @@ diag("tcp connect to $test->{host}:$test->{port} ok"); my $ctx = Net::SSLeay::CTX_new() or die "failed to create CTX"; - my $ssl = Net::SSLeay::new($ctx) or die "failed to create SSL"; # enable verification with hopefully usable CAs Net::SSLeay::CTX_set_default_verify_paths($ctx); @@ -75,10 +76,8 @@ if eval { require Mozilla::CA }; Net::SSLeay::CTX_set_verify($ctx,Net::SSLeay::VERIFY_PEER(),undef); - # setup TLS extension and callback to catch stapled OCSP response + # setup TLS extension callback to catch stapled OCSP response my $stapled_response; - Net::SSLeay::set_tlsext_status_type($ssl, - Net::SSLeay::TLSEXT_STATUSTYPE_ocsp()); Net::SSLeay::CTX_set_tlsext_status_cb($ctx,sub { my ($ssl,$resp) = @_; diag("got ".($resp ? '':'no ')."stapled OCSP response"); @@ -87,11 +86,21 @@ return 1; }); + # create SSL object only after we have the context fully done since + # some parts of the context (like verification mode) will be copied + # to the SSL object and thus later changes to the CTX don't affect + # the SSL object + my $ssl = Net::SSLeay::new($ctx) or die "failed to create SSL"; + + # setup TLS extension to request stapled OCSP response + Net::SSLeay::set_tlsext_status_type($ssl, + Net::SSLeay::TLSEXT_STATUSTYPE_ocsp()); + # non-blocking SSL_connect with timeout $cl->blocking(0); Net::SSLeay::set_fd($ssl,fileno($cl)); my $end = time() + $timeout; - my $rv; + my ($rv,@err); while (($rv = Net::SSLeay::connect($ssl)) < 0) { my $to = $end-time(); $to<=0 and last; @@ -101,10 +110,14 @@ select($vec,undef,undef,$to); } elsif ( $err == Net::SSLeay::ERROR_WANT_WRITE()) { select(undef,$vec,undef,$to); + } else { + while ( my $err = Net::SSLeay::ERR_get_error()) { + push @err, Net::SSLeay::ERR_error_string($err); + } + last } } - skip "SSL_connect with $test->{host}:$test->{port} failed: ". - Net::SSLeay::print_errs(''),1 + skip "SSL_connect with $test->{host}:$test->{port} failed: @err",1 if $rv<=0; diag("SSL_connect ok"); @@ -114,8 +127,11 @@ my $fp = $leaf_cert && unpack("H*",Net::SSLeay::X509_digest($leaf_cert,$sha1)); skip "could not get fingerprint",1 if !$fp; - skip "bad fingerprint $fp for $test->{host}:$test->{port}",1 - if $fp ne $test->{fingerprint}; + if ($fp ne $test->{fingerprint}) { + push @fp_mismatch, [ $fp,$test ]; + skip("bad fingerprint for $test->{host}:$test->{port} -". + " expected $test->{fingerprint}, got $fp",1) + } diag("fingerprint matches"); if ( $test->{ocsp_staple} && ! $stapled_response ) { @@ -225,6 +241,19 @@ } } +if ($release_tests) { + if (!@fp_mismatch) { + pass("all fingerprints matched"); + } else { + for(@fp_mismatch) { + my ($fp,$test) = @$_; + diag("fingerprint mismatch for $test->{host}:$test->{port} -". + " expected $test->{fingerprint}, got $fp") + } + fail("some fingerprints did not matched - please adjust test"); + } +} + { # cleanup stuff when going out of scope package __cleanup__;
Subject: Re: [rt.cpan.org #116795] OCSP broken in 1.75+ - patch included
Date: Sat, 13 Aug 2016 09:54:22 +1000
To: bug-Net-SSLeay [...] rt.cpan.org
From: Mike McCauley <mikem [...] airspayce.com>
Hi Steffen, Thanks Tests OK with: openssl-1.1.0 libressl-2.4.1 openssl-1.0.2c openssl-1.0.2a openssl-1.0.2 openssl-1.0.1c openssl-1.0.1 openssl-1.0.0d openssl-1.0.0 openssl-0.9.8i+extension I have committed to SVN. If you report that SVN tests OK for you I will release a new version to CPAN. Cheers. On Friday, August 12, 2016 03:49:16 AM you wrote: Show quoted text
> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=116795 > >
> > With openssl-1.1.0, openssl-1.0.2c etc I get: > > ... > > # Failed test 'failed to get OCSP_CERTIDs for cert /C=US/O=Symantec > > Corporation/OU=Symantec Trust Network/CN=Symantec Class 3 Secure > > Server CA - > > G4: cannot find issuer certificate at t/external/ocsp.t line 141.
> > This is hopefully solved now. > Inside the test I've first created the context and the SSL object and then > added the features to the context I need. It looks like that some features > in the SSL object are taken from the linked context while other features > are copied from the context on creation of the SSL object and further > changes to the context will not affect the SSL object. > > One of these copied features is the verification mode. Due to the order of > creation of SSL object and context the SSL object ended up with a mode > of VERIFY_NONE even if the context was set to VERIFY_PEER. And this caused > the SSL_connect to succeed even though the certificate chain could not be > verified against trusted CA. And because of the missing root CA the > OCSP_CERTID failed to get the issue certificate. > > This is now fixed by first creating and fully setting up the context before > creating the SSL object. This will probably mean that with your > openssl-1.1.0 etc setups the test will be skipped because the SSL > connections fail with "certificate verify failed". To fix this you might > need to install Mozilla::CA in your test environment since it looks like > that it cannot find the needed root CA.
-- Mike McCauley VK4AMM mikem@airspayce.com Airspayce Pty Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.airspayce.com Phone +61 7 5598-7474
Subject: Re: [rt.cpan.org #116795] OCSP broken in 1.75+ - patch included
Date: Sat, 13 Aug 2016 08:53:13 +0200
To: Mike McCauley via RT <bug-Net-SSLeay [...] rt.cpan.org>
From: Steffen Ullrich <Steffen_Ullrich [...] genua.de>
Show quoted text
> I have committed to SVN. If you report that SVN tests OK for you I will > release a new version to CPAN.
Looks good to me. I've tested with various OpenSSL and Perl versions on Ubuntu and with LibreSSL on OpenBSD 5.9 Tanks a lot, Steffen
Subject: Re: [rt.cpan.org #116795] OCSP broken in 1.75+ - patch included
Date: Sat, 13 Aug 2016 18:42:42 +1000
To: bug-Net-SSLeay [...] rt.cpan.org
From: Mike McCauley <mikem [...] airspayce.com>
Thanks Steffen, version 1.78 uploaded to CPAN. Cheers. On Saturday, August 13, 2016 02:53:42 AM Steffen Ullrich via RT wrote: Show quoted text
> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=116795 > >
> > I have committed to SVN. If you report that SVN tests OK for you I will > > release a new version to CPAN.
> > Looks good to me. > I've tested with various OpenSSL and Perl versions on Ubuntu and with > LibreSSL on OpenBSD 5.9 > > Tanks a lot, > Steffen
-- Mike McCauley VK4AMM mikem@airspayce.com Airspayce Pty Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.airspayce.com Phone +61 7 5598-7474