Skip Menu |

This queue is for tickets about the Net-SSLeay CPAN distribution.

Report information
The Basics
Id: 124360
Status: open
Priority: 0/
Queue: Net-SSLeay

People
Owner: Nobody in particular
Requestors: sludin [...] ludin.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 1.84
Fixed in: (no value)



Subject: Adding support for generating OCSP responses
Currently Net::SSLeay cannot be used to create an OCSP responder (at least as far as I can tell). I would to enhance it to be able to do that. When finished my plan is to submit a patch for potential inclusion. I have a horrible hack working now, which though functional would not (should not?) meet the bar for inclusion. Before I clean it up though I wanted to get some opinions for the maintainers. Essentially I want to replicate the functionality of make_ocsp_request in apps/ocsp.c. The first directional question is do I drop that call (and all of its dependencies) as a helper function into SSLeay.xs, or do I implement the 15 or so calls that it uses that are not already in Net::SSLeay and write make_ocsp_request in perl. The former has a bit junky on the inside, provides less flexibility, but makes the smallest change to the interface. The latter will add a bunch of calls that will then need to be documented and maintained, but provides tons of flexibility and expands the capabilities of Net::SSLeay beyond just OCSP responses (possibly). What is the preferred direction? By the way, is there a best practice for XS calls that need to allocate memory? I may need an OPENSSL_malloc, but I see that nowhere else in SSLeay.xs is a malloc made which leads me to think that may be something to avoid.
Subject: Re: [rt.cpan.org #124360] Adding support for generating OCSP responses
Date: Thu, 08 Feb 2018 06:51:14 +1000
To: bug-Net-SSLeay [...] rt.cpan.org
From: Mike McCauley <mikem [...] airspayce.com>
Hello Stephen, On Thursday, 8 February 2018 03:37:15 AEST you wrote: Show quoted text
> Wed Feb 07 12:37:14 2018: Request 124360 was acted upon. > Transaction: Ticket created by SLUDIN > Queue: Net-SSLeay > Subject: Adding support for generating OCSP responses > Broken in: 1.84 > Severity: Normal > Owner: Nobody > Requestors: sludin@ludin.org > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=124360 > > > > Currently Net::SSLeay cannot be used to create an OCSP responder (at least > as far as I can tell). I would to enhance it to be able to do that. When > finished my plan is to submit a patch for potential inclusion. I have a > horrible hack working now, which though functional would not (should not?) > meet the bar for inclusion. Before I clean it up though I wanted to get > some opinions for the maintainers. > > Essentially I want to replicate the functionality of make_ocsp_request in > apps/ocsp.c. The first directional question is do I drop that call (and > all of its dependencies) as a helper function into SSLeay.xs, or do I > implement the 15 or so calls that it uses that are not already in > Net::SSLeay and write make_ocsp_request in perl. The former has a bit > junky on the inside, provides less flexibility, but makes the smallest > change to the interface. The latter will add a bunch of calls that will > then need to be documented and maintained, but provides tons of flexibility > and expands the capabilities of Net::SSLeay beyond just OCSP responses > (possibly). > > What is the preferred direction?
I think most people would prefer to see the latter approach, though you are correct that it is more work. Show quoted text
> > By the way, is there a best practice for XS calls that need to allocate > memory? I may need an OPENSSL_malloc, but I see that nowhere else in > SSLeay.xs is a malloc made which leads me to think that may be something to > avoid.
No, thats fine: if you need mem you prob should use OPENSSL_malloc and friends. -- Mike McCauley VK4AMM mikem@airspayce.com Airspayce Pty Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.airspayce.com Phone +61 7 5598-7474
On Wed Feb 07 16:15:52 2018, mikem@airspayce.com wrote: Show quoted text
> Hello Stephen, > > On Thursday, 8 February 2018 03:37:15 AEST you wrote:
> > Wed Feb 07 12:37:14 2018: Request 124360 was acted upon. > > Transaction: Ticket created by SLUDIN > > Queue: Net-SSLeay > > Subject: Adding support for generating OCSP responses > > Broken in: 1.84 > > Severity: Normal > > Owner: Nobody > > Requestors: sludin@ludin.org > > Status: new > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=124360 > > > > > > > Currently Net::SSLeay cannot be used to create an OCSP responder (at > > least > > as far as I can tell). I would to enhance it to be able to do that. > > When > > finished my plan is to submit a patch for potential inclusion. I > > have a > > horrible hack working now, which though functional would not (should > > not?) > > meet the bar for inclusion. Before I clean it up though I wanted to > > get > > some opinions for the maintainers. > > > > Essentially I want to replicate the functionality of > > make_ocsp_request in > > apps/ocsp.c. The first directional question is do I drop that call > > (and > > all of its dependencies) as a helper function into SSLeay.xs, or do I > > implement the 15 or so calls that it uses that are not already in > > Net::SSLeay and write make_ocsp_request in perl. The former has a > > bit > > junky on the inside, provides less flexibility, but makes the > > smallest > > change to the interface. The latter will add a bunch of calls that > > will > > then need to be documented and maintained, but provides tons of > > flexibility > > and expands the capabilities of Net::SSLeay beyond just OCSP > > responses > > (possibly). > > > > What is the preferred direction?
> > I think most people would prefer to see the latter approach, though > you are > correct that it is more work.
I will try and do some variant of that. Digging in it looks like it would be good to spend some time to 'perlify' it rather than a literal translation. For example, making STACK_OFs into AV * or returning a hash from things like OCSP_id_get0_info. I'll come up with something and then let the experts kick it around. Show quoted text
>
> > > > By the way, is there a best practice for XS calls that need to > > allocate > > memory? I may need an OPENSSL_malloc, but I see that nowhere else in > > SSLeay.xs is a malloc made which leads me to think that may be > > something to > > avoid.
> > No, thats fine: if you need mem you prob should use OPENSSL_malloc and > friends.
Attached is an almost direct perl translation of make_ocsp_response from apps/ocsp.c. I would love feedback on the general look and feel. My concern is that as it stands it requires a ton of openssl knowledge to understand and properly use, but then I suppose that is no different than the rest of Net::SSLeay. Everything works and is implemented on the XS side. Is there a general guideline for when to return opaque openssl pointers vs. when to return perl objects. For example, I have Net::SSLeay::OCSP_id_get0_inf returning a hash ref but just about everything else returning internal pointers.
Subject: responder.pl
use Devel::Hide qw( Net/SSLeay.pm ); use strict; use warnings; use lib './blib/lib'; use lib './blib/arch'; use Net::SSLeay qw( die_now ); use Data::Dumper; use IO::File; use File::Slurp; use Devel::Peek; Net::SSLeay::load_error_strings(); Net::SSLeay::SSLeay_add_ssl_algorithms(); Net::SSLeay::randomize(); my $content = read_file( "req2.der" ); my $req = Net::SSLeay::d2i_OCSP_REQUEST( $content ); my $index = Net::SSLeay::load_index( "index.txt", 0 ); my $cert = load_cert( "servercert.pem" ) || die; my $ca = load_cert( "intermediatecert.pem" ) || die; my $rkey = load_key( "intermediatekey.pem" ) || die; my $resp = make_ocsp_response( $req, $index, $ca, $ca, $rkey, "sha1", [ $cert ], 0, 30, 3, 0 ); my $der = Net::SSLeay::i2d_OCSP_RESPONSE( $resp ); write_file( "resp.der", $der ); sub load_cert { my $filename = shift; $bio = Net::SSLeay::BIO_new_file( $filename, 'r' ) || die_now( "BIO_new_file ($!)" ); my $der = Net::SSLeay::PEM_read_bio_X509($bio) || die_now( "BIO_read_bio_X509 ($!)" );; Net::SSLeay::BIO_free($bio); return $der; } sub load_key { my $filename = shift; $bio = Net::SSLeay::BIO_new_file( $filename, 'r' ) || die_now( "BIO_new_file ($!)" ); my $der = Net::SSLeay::PEM_read_bio_PrivateKey($bio) || die_now( "BIO_read_bio_PrivateKey ($!)" );; Net::SSLeay::BIO_free($bio); return $der; } sub make_ocsp_response { my $req = shift; my $db = shift; my $ca = shift; my $rcert = shift; my $rkey = shift; my $md = shift || "sha1"; my $rother = shift; my $flags = shift; my $nmin = shift; my $ndays = shift; my $badsig = shift; my $resp; my $i; # Load the digest hash from openssl my $rmd = Net::SSLeay::EVP_get_digestbyname( $md ) || die_now( "EVP_get_digestbyname ($!)" ); # get the count of certs to check from the request my $id_count = Net::SSLeay::OCSP_request_onereq_count( $req ); if ( $id_count <= 0 ) { $resp = Net::SSLeay::OCSP_response_create( Net::SSLeay::OCSP_RESPONSE_STATUS_MALFORMEDREQUEST(), 0 ); return $resp; } # BASICRESP is where we will build the respone. The signed version of it will # constitute the final response my $bs = Net::SSLeay::OCSP_BASICRESP_new(); # create the thisUpdate and nextUpdate fields my $nextupd; my $thisupd = Net::SSLeay::X509_gmtime_adj( 0, 0 ); if ( $ndays != -1 ) { $nextupd = Net::SSLeay::X509_time_adj_ex( 0, $ndays, $nmin * 60, 0 ); } # handle each cert in the request for my $i ( 0 .. $id_count - 1 ) { # ONEREQ is the 'Request' structure from the ASN.1 ( RFC 6960 ) my $one = Net::SSLeay::OCSP_request_onereq_get0( $req, $i ); # the CertID contains the hashAlgorithm, issuerNameHash, issuerKeyHash, and the serialNumber # this three-tuple uniquely identifies the certificate my $cid = Net::SSLeay::OCSP_onereq_get0_id( $one ); # $info is a hash of the CertID data my $info = Net::SSLeay::OCSP_id_get0_info( $cid ); my $cert_id_md = Net::SSLeay::EVP_get_digestbyobj( $info->{hashAlgorithm} ); if ( ! $cert_id_md ) { $resp = Net::SSLeay::OCSP_response_create( OCSP_RESPONSE_STATUS_INTERNALERROR(), 0 ); print "No message digest\n"; return $resp; } # extract the CertID from the issuing cert my $ca_id = Net::SSLeay::OCSP_cert_to_id( $cert_id_md, 0, $ca ); # if the CertID from the request and the CertID from the issuer do not match # respond saying as much. if ( Net::SSLeay::OCSP_id_issuer_cmp( $ca_id, $cid ) ) { Net::SSLeay::OCSP_basic_add1_status( $bs, $cid, Net::SSLeay::V_OCSP_CERTSTATUS_UNKNOWN(), 0, 0, $thisupd, $nextupd); print "Here\n"; next; } # Lookup the serial in the CA index my $inf = Net::SSLeay::lookup_serial( $db, $info->{serialNumber} ); if ( ! $inf ) { # Can't find it. Respond as unkonwn Net::SSLeay::OCSP_basic_add1_status( $bs, $cid, Net::SSLeay::V_OCSP_CERTSTATUS_UNKNOWN(), 0, 0, $thisupd, $nextupd); } # TODO: replace with constants DB_type and DB_TYPE_ elsif ( $inf->[0] eq "V" ) { # Found it and it is good Net::SSLeay::OCSP_basic_add1_status( $bs, $cid, Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), 0, 0, $thisupd, $nextupd); } # TODO: replace with constants DB_type and DB_TYPE_REV elsif( $inf->[0] eq "R" ) { # Found it and it is revoked Net::SSLeay::OCSP_basic_add1_status( $bs, $cid, Net::SSLeay::V_OCSP_CERTSTATUS_UNKNOWN(), 0, 0, $thisupd, $nextupd); # unpack_revinfo(&revtm, &reason, &inst, &invtm, inf[DB_rev_date]); # single = OCSP_basic_add1_status(bs, cid, # V_OCSP_CERTSTATUS_REVOKED, # reason, revtm, thisupd, nextupd); # if (invtm) # OCSP_SINGLERESP_add1_ext_i2d(single, NID_invalidity_date, # invtm, 0, 0); # else if (inst) # OCSP_SINGLERESP_add1_ext_i2d(single, # NID_hold_instruction_code, inst, # 0, 0); # ASN1_OBJECT_free(inst); # ASN1_TIME_free(revtm); # ASN1_GENERALIZEDTIME_free(invtm); # } } else { # An unexpected status. Respond and Unknown; Net::SSLeay::OCSP_basic_add1_status( $bs, $cid, Net::SSLeay::V_OCSP_CERTSTATUS_UNKNOWN(), 0, 0, $thisupd, $nextupd); } # TODO: Oh, there are probably a bunch lf leaks right now. Net::SSLeay::OCSP_CERTID_free( $ca_id ); } # If the request had a nonce, copy it into the response Net::SSLeay::OCSP_copy_nonce( $bs, $req ); # Sign the BASICRESP # TODO: describe $rother Net::SSLeay::OCSP_basic_sign( $bs, $rcert, $rkey, $rmd, $rother, $flags ); # Purposely corrupt the signature if asked to. This is in the openssl # make_ocsp_response in apps/ocsp.c if ( $badsig ) { my $sig = Net::SSLeay::OCSP_resp_get0_signature( $bs ); Net:SSLeay::corrupt_signature( $sig ); } # create the successful response from the signed BASICRESP $resp = Net::SSLeay::OCSP_response_create( Net::SSLeay::OCSP_RESPONSE_STATUS_SUCCESSFUL(), $bs ); return $resp; }
Finally getting back to this. Code is written but needs some style work. One of the things I need is access to the structures in crypto/ocsp/ocsp_lcl.h. These structures are not put into the public include files. What is the best way to deal with this? Copy/Pasting them into SSLeay.xs work, but seems heavy handed. Any guidance is welcome.
Attached is an initial diff. This provides the basic needed functionality, attempts to conform to style, compiles and passes test on 0.9.8 (not functional), 1.0.2, and 1.1.0. It is a rather large patch. I am also attacking a test file. There is still a bit more work needed to finish it up, but I'd like to get feedback before I start the fine polish.
Subject: 70_ocsp_responder.t
#!/usr/bin/perl use strict; use warnings; use Test::More; use Socket; use File::Spec; use Symbol qw(gensym); use Net::SSLeay; use Config; BEGIN { plan skip_all => "openssl 1.0.2 required" unless Net::SSLeay::SSLeay >= 0x10002000; } plan tests => 10; my $sock; my $pid; my $port = 40000+int(rand(9999)); my $ip = "\x7F\0\0\x01"; my $serv_params = sockaddr_in($port, $ip); my $msg = 'ssleay-ocsp-responder-test'; my $intermediate_cert_pem = File::Spec->catfile('t', 'data', 'ocsptest_intermediate.crt.pem'); #my $intermediate_key_pem = File::Spec->catfile('t', 'data', 'ocsptest_intermediate.key.pem'); my $intermediate_ocsp_cert_pem = File::Spec->catfile('t', 'data', 'ocsptest_intermediate_ocsp.crt.pem'); my $intermediate_ocsp_key_pem = File::Spec->catfile('t', 'data', 'ocsptest_intermediate_ocsp.key.pem'); #my $server_cert_pem = File::Spec->catfile('t', 'data', 'ocsptest_server.crt.pem'); #my $server_key_pem = File::Spec->catfile('t', 'data', 'ocsptest_server.key.pem'); #my $root_cert_pem = File::Spec->catfile('t', 'data', 'ocsptest_root.crt.pem'); #my $root_key_pem = File::Spec->catfile('t', 'data', 'ocsptest_root.key.pem'); my $ca_index = File::Spec->catfile('t', 'data', 'ocsptest_index.txt'); my @results; Net::SSLeay::initialize(); { my $reqfile = File::Spec->catfile('t', 'data', 'ocsptest_req_nononce.der'); ok( $reqfile ); my $content = slurp( $reqfile ); ok( $content ); my $req = Net::SSLeay::d2i_OCSP_REQUEST( $content ); ok( $req ); my $index = Net::SSLeay::load_index( $ca_index, 0 ); ok ( $index ); my $intermediate_cert = load_cert( $intermediate_cert_pem ) || die; ok( $intermediate_cert ); # my $intermediate_key = load_key( $intermediate_key_pem ) || die; # ok( $intermediate_key ); my $ocsp_key = load_key( $intermediate_ocsp_key_pem ) || die; ok( $ocsp_key ); my $ocsp_cert = load_cert( $intermediate_ocsp_cert_pem ) || die; ok( $ocsp_cert ); # my $server_cert = load_cert( $server_cert_pem ) || die; # ok( $server_cert ); my $resp = make_ocsp_response( $req, $index, $intermediate_cert, $ocsp_cert, $ocsp_key, "sha1", [ ], 0, 30, 3, 0 ); ok( $resp ); my $der = Net::SSLeay::i2d_OCSP_RESPONSE( $resp ); ok( $der ); open( FILE, ">", "resp.der" ); print FILE $der; close( FILE ); } push @results, [$? == 0, 'server exited with 0']; END { # Test::More->builder->current_test(3); # ok( $_->[0], $_->[1] ) for (@results); } sub load_cert { my $filename = shift; my $bio = Net::SSLeay::BIO_new_file( $filename, 'r' ) || die_now( "BIO_new_file ($!)" ); my $der = Net::SSLeay::PEM_read_bio_X509($bio) || die_now( "BIO_read_bio_X509 ($!)" );; Net::SSLeay::BIO_free($bio); return $der; } sub load_key { my $filename = shift; my $bio = Net::SSLeay::BIO_new_file( $filename, 'r' ) || die_now( "BIO_new_file ($!)" ); my $der = Net::SSLeay::PEM_read_bio_PrivateKey($bio) || die_now( "BIO_read_bio_PrivateKey ($!)" );; Net::SSLeay::BIO_free($bio); return $der; } sub slurp { my $filename = shift; local $/; open( my $fh, '<', $filename ); my $text = <$fh>; return $text; } sub make_ocsp_response { my $req = shift; my $db = shift; my $ca = shift; my $rcert = shift; my $rkey = shift; my $md = shift || "sha1"; my $rother = shift; my $flags = shift; my $nmin = shift; my $ndays = shift; my $badsig = shift || 0; my $resp; my $i; # Load the digest hash from openssl my $rmd = Net::SSLeay::EVP_get_digestbyname( $md ) || die_now( "EVP_get_digestbyname ($!)" ); # get the count of certs to check from the request my $id_count = Net::SSLeay::OCSP_request_onereq_count( $req ); if ( $id_count <= 0 ) { $resp = Net::SSLeay::OCSP_response_create( Net::SSLeay::OCSP_RESPONSE_STATUS_MALFORMEDREQUEST(), 0 ); return $resp; } # BASICRESP is where we will build the respone. The signed version of it will # constitute the final response my $bs = Net::SSLeay::OCSP_BASICRESP_new(); # create the thisUpdate and nextUpdate fields my $nextupd; my $thisupd = Net::SSLeay::X509_gmtime_adj( 0, 0 ); if ( $ndays != -1 ) { $nextupd = Net::SSLeay::X509_time_adj_ex( 0, $ndays, $nmin * 60, 0 ); } # handle each cert in the request for my $i ( 0 .. $id_count - 1 ) { # ONEREQ is the 'Request' structure from the ASN.1 ( RFC 6960 ) my $one = Net::SSLeay::OCSP_request_onereq_get0( $req, $i ); # the CertID contains the hashAlgorithm, issuerNameHash, issuerKeyHash, and the serialNumber # this three-tuple uniquely identifies the certificate my $cid = Net::SSLeay::OCSP_onereq_get0_id( $one ); # $info is a hash of the CertID data my $info = Net::SSLeay::OCSP_id_get0_info( $cid ); my $cert_id_md = Net::SSLeay::EVP_get_digestbyobj( $info->{hashAlgorithm} ); if ( ! $cert_id_md ) { $resp = Net::SSLeay::OCSP_response_create( OCSP_RESPONSE_STATUS_INTERNALERROR(), 0 ); return $resp; } # extract the CertID from the issuing cert my $ca_id = Net::SSLeay::OCSP_cert_to_id( $cert_id_md, 0, $ca ); # if the CertID from the request and the CertID from the issuer do not match # respond saying as much. if ( Net::SSLeay::OCSP_id_issuer_cmp( $ca_id, $cid ) ) { Net::SSLeay::OCSP_basic_add1_status( $bs, $cid, Net::SSLeay::V_OCSP_CERTSTATUS_UNKNOWN(), 0, 0, $thisupd, $nextupd); next; } # Lookup the serial in the CA index my $inf = Net::SSLeay::lookup_serial( $db, $info->{serialNumber} ); if ( ! $inf ) { # Can't find it. Respond as unkonwn Net::SSLeay::OCSP_basic_add1_status( $bs, $cid, Net::SSLeay::V_OCSP_CERTSTATUS_UNKNOWN(), 0, 0, $thisupd, $nextupd); } # TODO: replace with constants DB_type and DB_TYPE_ elsif ( $inf->[0] eq "V" ) { # Found it and it is good Net::SSLeay::OCSP_basic_add1_status( $bs, $cid, Net::SSLeay::V_OCSP_CERTSTATUS_GOOD(), 0, 0, $thisupd, $nextupd); } # TODO: replace with constants DB_type and DB_TYPE_REV elsif( $inf->[0] eq "R" ) { # Found it and it is revoked Net::SSLeay::OCSP_basic_add1_status( $bs, $cid, Net::SSLeay::V_OCSP_CERTSTATUS_UNKNOWN(), 0, 0, $thisupd, $nextupd); my $arr = Net::SSLeay::unpack_revinfo( $inf->[2] ); Dump( $arr, 10 ); # unpack_revinfo(&revtm, &reason, &inst, &invtm, inf[DB_rev_date]); my $single = Net::SSLeay::OCSP_basic_add1_status( $bs, $cid, Net::SSLeay::V_OCSP_CERTSTATUS_REVOKED(), $arr->[1], $arr->[0], $thisupd, $nextupd); # TODO: Not 100% certain this is all working properly if ( $arr->[3] ) { Net::SSLeay::OCSP_SINGLERESP_add1_ext_i2d( $single, Net::SSLeay::NID_invalidity_date(), $arr->[3], 0, 0); } elsif ( $arr->[2] ) { Net::SSLeay::OCSP_SINGLERESP_add1_ext_i2d( $single, Net::SSLeay::NID_hold_instruction_code(), $arr->[2], 0, 0); } # TODO: Free $arr } else { # An unexpected status. Respond and Unknown; Net::SSLeay::OCSP_basic_add1_status( $bs, $cid, Net::SSLeay::V_OCSP_CERTSTATUS_UNKNOWN(), 0, 0, $thisupd, $nextupd); print "No status match\n"; } # TODO: Oh, there are probably a bunch lf leaks right now. Net::SSLeay::OCSP_CERTID_free( $ca_id ); } # If the request had a nonce, copy it into the response Net::SSLeay::OCSP_copy_nonce( $bs, $req ); # Sign the BASICRESP # TODO: describe $rother Net::SSLeay::OCSP_basic_sign( $bs, $rcert, $rkey, $rmd, $rother, $flags ); # Purposely corrupt the signature if asked to. This is in the openssl # make_ocsp_response in apps/ocsp.c if ( $badsig ) { my $sig = Net::SSLeay::OCSP_resp_get0_signature( $bs ); Net:SSLeay::corrupt_signature( $sig ); } # create the successful response from the signed BASICRESP $resp = Net::SSLeay::OCSP_response_create( Net::SSLeay::OCSP_RESPONSE_STATUS_SUCCESSFUL(), $bs ); return $resp; }
Subject: ocsp_responder.diff

Message body is not shown because it is too large.

On Thu Mar 08 20:06:19 2018, SLUDIN wrote: Show quoted text
> There is still a bit more work needed to finish it up, but I'd like to > get feedback before I start the fine polish.
Would you be interested in continuing the work? Now that we have a stable release behind us, it's a good time to look at new features. The patch still applies but because the attachments did not include the certificates, the test fails to run. If you still have the certs, could you attach the certs to this ticket? We could next look at topics such as function naming (/4 under naming conventions in SSLeay.xs) and how to handle special functionality such as corrupt_signature(). -- Heikki Vatiainen