Skip Menu |

This queue is for tickets about the Net-FTPSSL CPAN distribution.

Report information
The Basics
Id: 101388
Status: resolved
Priority: 0/
Queue: Net-FTPSSL

People
Owner: Nobody in particular
Requestors: arthurdent99 [...] hotmail.com
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 0.25
Fixed in: 0.26



Subject: SSL_reuse_ctx is missing certain options
I am submitting a patch to Net::FTPSSL to correct an issue when using the SSL_Client_Certificate option to trigger both the SSL_reuse_ctx option and the certificate and hostname verification options of IO::Socket::SSL. A particular vendor that I have to deal with runs an FTPS server of unknown origin. (They've overridden the connect messages, so I can't tell what program their server is running.) I have a Perl script that automatically uploads a file to their server. They recently changed their server settings due to the POODLE vulnerability, and they only support TLS ciphers now. Once their server settings changed, the upload stopped functioning. My script could log in to the site, but as soon as it opened the data channel to upload the file, the connection was dropped by their server immediately. In the initial connection settings, I am forcing a TLS connection by setting useSSL => 0. When the script first logged in, it did indeed do a TLS handshake and connect to the server just fine, with my client offering 20 TLS ciphers for the server to choose from. When it came time to open the data channel to transfer the file, however, the data channel reverted to an SSL handshake instead of a TLS handshake, and my client offered 26 ciphers instead of the original 20 ciphers, with the extra 6 ciphers being all SSLv2 ciphers for some reason (not even SSLv3). Their server settings are set to reject SSL ciphers now, so as soon as their server saw the extra old SSLv2 ciphers, they dropped the connection. I eventually found the setting in your module for SSL_Client_Certificate. Even using an empty hash causes your module to use the SSL_reuse_ctx option in IO::Socket::SSL. As soon as I set this value, then the script started to work properly again. The data channel reused the SSL context of the command channel, so it did a TLS handshake and offered the same 20 ciphers that the command channel did, instead of the behavior mentioned above. By itself, this works great, and it solved my issue. When I discovered that no hostname verification was going on, however, I added the IO::Socket::SSL settings for hostname verification, and that's when things started to break. Apparently, reusing the SSL context does not reuse all of the initial settings, but only "context-related" settings, whatever those are (according to the IO::Socket::SSL documentation). Also apparently, the settings for SSL_ca_path, SSL_verifycn_name, and SSL_verifycn_scheme (which I am using for hostname verification) are not included in the "context-related" settings. So while the context is indeed being reused for the data channel, by dropping the path to the CA certificate file, the data channel connection fails because it cannot properly verify the certificate anymore. According to the IO::Socket::SSL documentation, you can send additional settings when reusing an SSL context, but any settings inside the context itself will override any duplicate settings also sent separately, so there should be no danger from sending a setting twice. And since the SSL_ca_path, SSL_verifycn_name, and SSL_verifycn_scheme settings are not inside the SSL context, those need to be re-sent separately when opening the data channel or else they will not be present at all. Also, when your module prints an error message inside _get_data_channel, it was trying to print the contents of the variable $ssl_opts{SSL_version}, but this variable is not present when reusing an SSL context. To fix everything, I added a simple patch to your code. If the arguments for SSL_version, SSL_ca_path, SSL_verifycn_name, and SSL_verifycn_scheme are present, they are added into the %ssl_reuse hash used for context reuse, so that they will be present if the context is indeed reused later. To test everything out, I set up my own FTPS-I server internally, running vsftpd 2.2.2 on Red Hat Enterprise Linux 6. I created my own SSL certificate for the server, using my company's private certificate authority, and I added this private CA certificate to the certificate bundle used by my Perl script so that it would validate successfully. Once I applied the simple patch to your code, then both certificate verification and hostname verification worked properly, and the data channel opened successfully and transferred the file. Without the patch, the command channel would open successfully, but certificate verification would fail when opening the data channel because IO::Socket::SSL had lost the path to the CA certificate file and the settings for hostname verification were also missing. FYI, the server I was running the script on was Red Hat Enterprise Linux 5.11, with Perl 5.8.8, Net::FTPSSL 0.25, and IO::Socket::SSL 2.008. Thanks, Chris Thompson
Subject: Net-FTPSSL-0.25.patch
diff -rupN Net-FTPSSL-0.25/FTPSSL.pm Net-FTPSSL-0.25-new/FTPSSL.pm --- Net-FTPSSL-0.25/FTPSSL.pm 2014-09-05 13:24:16.000000000 -0500 +++ Net-FTPSSL-0.25-new/FTPSSL.pm 2015-01-06 18:52:42.000000000 -0600 @@ -352,6 +352,18 @@ sub new { (ref ($arg->{SSL_Advanced}) eq "HASH") ) { # Reuse the command channel context ... my %ssl_reuse = ( SSL_reuse_ctx => ${*$obj}{_SSL_ctx} ); + if (defined($ssl_args{SSL_version})) { + $ssl_reuse{SSL_version} = $ssl_args{SSL_version}; + } + if (defined($ssl_args{SSL_ca_file})) { + $ssl_reuse{SSL_ca_file} = $ssl_args{SSL_ca_file}; + } + if (defined($ssl_args{SSL_verifycn_name})) { + $ssl_reuse{SSL_verifycn_name} = $ssl_args{SSL_verifycn_name}; + } + if (defined($ssl_args{SSL_verifycn_scheme})) { + $ssl_reuse{SSL_verifycn_scheme} = $ssl_args{SSL_verifycn_scheme}; + } ${*$obj}{myContext} = \%ssl_reuse; }
Hi Chris, Thanks for your research & patch. Going through IO::Socket::SSL to figure things out is always a pain. Especially if I'm not allowed to connect to the problem server to try things out. There are just too many variables involved. I'll see if I can get a beta ready for you to try out in the next couple of days. I'm assuming your patch goes towards the end of function "new()". I'd also appreciate it if you could turn on the logs so that I can follow the trace of what is happening to help me better understand the issues you are hitting. Both a before & after trace would help me out the most. Curtis On Wed Jan 07 19:07:21 2015, christhompson wrote: Show quoted text
> I am submitting a patch to Net::FTPSSL to correct an issue when using > the SSL_Client_Certificate option to trigger both the SSL_reuse_ctx > option and the certificate and hostname verification options of > IO::Socket::SSL. >
...... Show quoted text
> > Thanks, > > Chris Thompson
Hi Chris, Here's my promised patch for you to test out. It's slightly different from what you provided. But I'm hopping it will still work for you. The other changes were all minor and doesn't affect this functionality. Please give me some feedback on whether or not it works for you along with the logs. Once I hear it works, I'll look into making it a formal release in the next week or so. Curtis On Wed Jan 07 20:38:42 2015, CLEACH wrote: Show quoted text
> Hi Chris, > > Thanks for your research & patch. Going through IO::Socket::SSL to > figure things out is always a pain. Especially if I'm not allowed to > connect to the problem server to try things out. There are just too > many variables involved. > > I'll see if I can get a beta ready for you to try out in the next > couple of days. > > I'm assuming your patch goes towards the end of function "new()". > > I'd also appreciate it if you could turn on the logs so that I can > follow the trace of what is happening to help me better understand the > issues you are hitting. Both a before & after trace would help me out > the most. > > Curtis > > On Wed Jan 07 19:07:21 2015, christhompson wrote:
> > I am submitting a patch to Net::FTPSSL to correct an issue when using > > the SSL_Client_Certificate option to trigger both the SSL_reuse_ctx > > option and the certificate and hostname verification options of > > IO::Socket::SSL. > >
> ......
> > > > Thanks, > > > > Chris Thompson
Chris, Did you have a chance to test out my patch? I did a bit more reading over the weekend and it's beginning to look like all 4 variables should have been included as part of the context. Some of the older versions of IO-Socket-SSL lists what options make up the context, and say that adding these options will be ignored, while the latest release suggests that it overrides the context but doesn't list what makes up the context. Confusing. So I'll have to do a bit more reading of the code to figure it out. In my latest beta, I dropped SSL_version from the list since you implied you only added it to remove a warning in my code. So I fixed my code to get the SSL_version elsewhere since it was only needed for an error message. Curtis On Fri Jan 09 20:18:43 2015, CLEACH wrote: Show quoted text
> Hi Chris, > > Here's my promised patch for you to test out. It's slightly different > from what you provided. But I'm hopping it will still work for you. > The other changes were all minor and doesn't affect this > functionality. > > Please give me some feedback on whether or not it works for you along > with the logs. Once I hear it works, I'll look into making it a > formal release in the next week or so. > > Curtis > > On Wed Jan 07 20:38:42 2015, CLEACH wrote:
> > Hi Chris, > > > > Thanks for your research & patch. Going through IO::Socket::SSL to > > figure things out is always a pain. Especially if I'm not allowed to > > connect to the problem server to try things out. There are just too > > many variables involved. > > > > I'll see if I can get a beta ready for you to try out in the next > > couple of days. > > > > I'm assuming your patch goes towards the end of function "new()". > > > > I'd also appreciate it if you could turn on the logs so that I can > > follow the trace of what is happening to help me better understand > > the > > issues you are hitting. Both a before & after trace would help me > > out > > the most. > > > > Curtis > > > > On Wed Jan 07 19:07:21 2015, christhompson wrote:
> > > I am submitting a patch to Net::FTPSSL to correct an issue when > > > using > > > the SSL_Client_Certificate option to trigger both the SSL_reuse_ctx > > > option and the certificate and hostname verification options of > > > IO::Socket::SSL. > > >
> > ......
> > > > > > Thanks, > > > > > > Chris Thompson
Chris, Did you have a chance to test out my patch? I did a bit more reading over the weekend and it's beginning to look like all 4 variables should have been included as part of the context. Some of the older versions of IO-Socket-SSL lists what options make up the context, and say that adding these options will be ignored, while the latest release suggests that it overrides the context but doesn't list what makes up the context. Confusing. So I'll have to do a bit more reading of the code to figure it out. In my latest beta, I dropped SSL_version from the list since you implied you only added it to remove a warning in my code. So I fixed my code to get the SSL_version elsewhere since it was only needed for an error message. Curtis On Fri Jan 09 20:18:43 2015, CLEACH wrote: Show quoted text
> Hi Chris, > > Here's my promised patch for you to test out. It's slightly different > from what you provided. But I'm hopping it will still work for you. > The other changes were all minor and doesn't affect this > functionality. > > Please give me some feedback on whether or not it works for you along > with the logs. Once I hear it works, I'll look into making it a > formal release in the next week or so. > > Curtis > > On Wed Jan 07 20:38:42 2015, CLEACH wrote:
> > Hi Chris, > > > > Thanks for your research & patch. Going through IO::Socket::SSL to > > figure things out is always a pain. Especially if I'm not allowed to > > connect to the problem server to try things out. There are just too > > many variables involved. > > > > I'll see if I can get a beta ready for you to try out in the next > > couple of days. > > > > I'm assuming your patch goes towards the end of function "new()". > > > > I'd also appreciate it if you could turn on the logs so that I can > > follow the trace of what is happening to help me better understand > > the > > issues you are hitting. Both a before & after trace would help me > > out > > the most. > > > > Curtis > > > > On Wed Jan 07 19:07:21 2015, christhompson wrote:
> > > I am submitting a patch to Net::FTPSSL to correct an issue when > > > using > > > the SSL_Client_Certificate option to trigger both the SSL_reuse_ctx > > > option and the certificate and hostname verification options of > > > IO::Socket::SSL. > > >
> > ......
> > > > > > Thanks, > > > > > > Chris Thompson
Chris, Did you have a chance to test out my patch? I did a bit more reading over the weekend and it's beginning to look like all 4 variables should have been included as part of the context. Some of the older versions of IO-Socket-SSL lists what options make up the context, and say that adding these options will be ignored, while the latest release suggests that it overrides the context but doesn't list what makes up the context. Confusing. So I'll have to do a bit more reading of the code to figure it out. In my latest beta, I dropped SSL_version from the list since you implied you only added it to remove a warning in my code. So I fixed my code to get the SSL_version elsewhere since it was only needed for an error message. Curtis On Fri Jan 09 20:18:43 2015, CLEACH wrote: Show quoted text
> Hi Chris, > > Here's my promised patch for you to test out. It's slightly different > from what you provided. But I'm hopping it will still work for you. > The other changes were all minor and doesn't affect this > functionality. > > Please give me some feedback on whether or not it works for you along > with the logs. Once I hear it works, I'll look into making it a > formal release in the next week or so. > > Curtis > > On Wed Jan 07 20:38:42 2015, CLEACH wrote:
> > Hi Chris, > > > > Thanks for your research & patch. Going through IO::Socket::SSL to > > figure things out is always a pain. Especially if I'm not allowed to > > connect to the problem server to try things out. There are just too > > many variables involved. > > > > I'll see if I can get a beta ready for you to try out in the next > > couple of days. > > > > I'm assuming your patch goes towards the end of function "new()". > > > > I'd also appreciate it if you could turn on the logs so that I can > > follow the trace of what is happening to help me better understand > > the > > issues you are hitting. Both a before & after trace would help me > > out > > the most. > > > > Curtis > > > > On Wed Jan 07 19:07:21 2015, christhompson wrote:
> > > I am submitting a patch to Net::FTPSSL to correct an issue when > > > using > > > the SSL_Client_Certificate option to trigger both the SSL_reuse_ctx > > > option and the certificate and hostname verification options of > > > IO::Socket::SSL. > > >
> > ......
> > > > > > Thanks, > > > > > > Chris Thompson
From: arthurdent99 [...] hotmail.com
Hello, Sorry it's taken me a few days to get to this. Things have been pretty busy at work. Your patch was not attached to the email that I received, and I don't see your patch attached on the website either. If you want to try to re-post your patch on the website, or if you want to email it to me directly, then I'll download it and give it a try. Thanks, Chris Thompson On Mon Jan 12 20:48:49 2015, CLEACH wrote: Show quoted text
> Chris, > > Did you have a chance to test out my patch?
Subject: RE: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options
Date: Wed, 14 Jan 2015 19:59:03 +0000
To: "bug-Net-FTPSSL [...] rt.cpan.org" <bug-Net-FTPSSL [...] rt.cpan.org>
From: Curtis Leach <cleach [...] Caesars.com>
Here it is again. The latest beta Curtis Show quoted text
-----Original Message----- From: Chris Thompson via RT [mailto:bug-Net-FTPSSL@rt.cpan.org] Sent: Wednesday, January 14, 2015 11:38 AM Subject: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options Queue: Net-FTPSSL Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=101388 > Hello, Sorry it's taken me a few days to get to this. Things have been pretty busy at work. Your patch was not attached to the email that I received, and I don't see your patch attached on the website either. If you want to try to re-post your patch on the website, or if you want to email it to me directly, then I'll download it and give it a try. Thanks, Chris Thompson On Mon Jan 12 20:48:49 2015, CLEACH wrote:
> Chris, > > Did you have a chance to test out my patch?

Message body is not shown because sender requested not to inline it.

From: arthurdent99 [...] hotmail.com
Thank you for your patch. In the patch I sent, I was checking for $ssl_args{SSL_verifycn_name}, etc. In the patch that you sent, you changed it to ${*$obj}{SSL_verifycn_name}, etc. But those options are not directly within the main object, they are inside a hash called _SSL_arguments. So, if you change: ${*$obj}{SSL_verifycn_name} to ${*$obj}{_SSL_arguments}{SSL_verifycn_name} and so on, then it would almost work. The only reason it doesn't quite work is that the value for SSL_verifycn_scheme isn't going into the _SSL_arguments hash for some reason. All the other options are there, but SSL_verifycn_scheme is missing, and I don't know why. Without that option changed from the default, then the strict name checking simply doesn't happen. That's why I pulled the values from the $ssl_args instead, just to pull that missing value for SSL_verifycn_scheme. Since the PASV command could potentially return a different IP address for the data channel, then not having the SSL_verifycn_scheme option carried over to the data channel means that the strict name checking wouldn't happen on that other IP address. I've attached what the screen looks like when I turn on debugging and transfer a file (after inserting the {_SSL_arguments} hash to your patch as I mentioned above). The transfer actually happens, but the SSL_verifycn_scheme is not passed to the data channel, because it's not present in the _SSL_arguments hash for some reason. If I leave the code as it was originally when I sent my patch, then you can see the SSL_verifycn_scheme option within the myContext hash, so I know it's being used for the data channel when looking under $ssl_args instead. Please let me know if you have any questions or need anything else. Thanks, Chris Thompson ----- On Wed Jan 14 14:59:19 2015, cleach@Caesars.com wrote: Show quoted text
> Here it is again. The latest beta > > Curtis
Subject: FTPSSL-debug.txt
Net-FTPSSL Version: 0.26 Perl: 5.008008 [5.8.8], OS: linux Server (port): testserver.testdomain.com (990) Keys: (OverridePASV), (Debug), (Encryption), (OverrideHELP), (Port), (SSL_Client_Certificate), (useSSL) Values: (), (1), (I), (ARRAY(0x9cd7094)), (), (HASH(0x9cd7340)), (0) <<< 220 (vsFTPd 2.2.2) Object Net::FTPSSL Details ... (testserver.testdomain.com:990 - I) Croak ==> (undef) Crypt ==> I FixGetTs ==> 0 FixPutTs ==> 0 Host ==> testserver.testdomain.com OverridePASV ==> Pret ==> 0 Timeout ==> 120 _SSL_arguments ==> HASH(0x9dc1a10) -- PeerAddr ===> 10.10.66.188 -- PeerPort ===> 990 -- Proto ===> tcp -- SSL_ca_file ===> /etc/pki/tls/certs/ca-bundle.crt -- SSL_check_crl ===> 0 -- SSL_cipher_list ===> ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES128-SHA256 ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES256-SHA384 ECDHE-ECDSA-AES128-SHA ECDHE-ECDSA-AES256-SHA ECDHE-RSA-AES128-SHA256 ECDHE-RSA-AES128-SHA ECDHE-RSA-AES256-SHA DHE-DSS-AES128-SHA256 DHE-DSS-AES128-SHA DHE-DSS-AES256-SHA256 DHE-DSS-AES256-SHA AES128-SHA256 AES128-SHA AES256-SHA256 AES256-SHA EDH-DSS-DES-CBC3-SHA DES-CBC3-SHA RC4-SHA !EXP !LOW !eNULL !aNULL !DES !MD5 !PSK !SRP -- SSL_server ===> 0 -- SSL_use_cert ===> 0 -- SSL_verify_callback ===> CODE(0x9dc1a34) -- SSL_verify_mode ===> 1 -- SSL_verifycn_name ===> testserver.testdomain.com -- SSL_version ===> TLSv1 _SSL_ctx ==> IO::Socket::SSL::SSL_Context=HASH(0x9dc1af4) -- context ===> 165381768 -- ocsp_mode ===> 16 -- verify_mode ===> 1 -- verify_name_ref ===> SCALAR(0x9b28808) _SSL_fileno ==> 4 _SSL_ioclass_upgraded ==> IO::Socket::INET _SSL_last_err ==> SSL wants a read first _SSL_object ==> 165878256 _SSL_opened ==> 1 buf_size ==> 10240 data_prot ==> P dcsc_mode ==> 1 debug ==> 1 debug_extra ==> 0 help_SITE_msg ==> 214 HELP Command Overriden by request. help_cmds_found ==> HASH(0x9cbe0e4) -- ABOR ===> 1 -- ACCT* ===> 1 -- ALLO ===> 1 -- APPE ===> 1 -- AUTH ===> 1 -- CCC ===> 1 -- CDUP ===> 1 -- CWD ===> 1 -- DELE ===> 1 -- HELP ===> 1 -- LIST ===> 1 -- MAIL* ===> 1 -- MDTM ===> 1 -- MKD ===> 1 -- MLFL* ===> 1 -- MODE ===> 1 -- MRCP* ===> 1 -- MRSQ* ===> 1 -- MSAM* ===> 1 -- MSND* ===> 1 -- MSOM* ===> 1 -- NLST ===> 1 -- NOOP ===> 1 -- PASS ===> 1 -- PASV ===> 1 -- PBSZ ===> 1 -- PORT ===> 1 -- PROT ===> 1 -- PWD ===> 1 -- QUIT ===> 1 -- REIN* ===> 1 -- REST ===> 1 -- RETR ===> 1 -- RMD ===> 1 -- RNFR ===> 1 -- RNTO ===> 1 -- SITE ===> 1 -- SIZE ===> 1 -- SMNT* ===> 1 -- STAT ===> 1 -- STOR ===> 1 -- STOU ===> 1 -- STRU ===> 1 -- SYST ===> 1 -- TYPE ===> 1 -- USER ===> 1 -- XCUP ===> 1 -- XCWD ===> 1 -- XMKD ===> 1 -- XPWD ===> 1 -- XRMD ===> 1 help_cmds_msg ==> 214 HELP Command Overriden by request. help_cmds_no_syntax_available ==> 1 io_socket_domain ==> 2 io_socket_proto ==> 6 io_socket_timeout ==> 120 io_socket_type ==> 1 last_ftp_msg ==> 220 (vsFTPd 2.2.2) myContext ==> HASH(0x9cbe2a0) -- SSL_ca_file ===> /etc/pki/tls/certs/ca-bundle.crt -- SSL_reuse_ctx ===> IO::Socket::SSL::SSL_Context=HASH(0x9dc1af4) -- context ----> 165381768 -- ocsp_mode ----> 16 -- verify_mode ----> 1 -- verify_name_ref ----> SCALAR(0x9b28808) -- SSL_verifycn_name ===> testserver.testdomain.com mySocketOpts ==> HASH(0x9c2f834) -- PeerAddr ===> testserver.testdomain.com -- PeerPort ===> 990 -- Proto ===> tcp -- Timeout ===> 120 trace ==> 0 type ==> A >>> USER +++++++ <<< 331 Please specify the password. >>> PASS ******* <<< 230 Login successful. <<+ 214 The HELP command is supported. >>> TYPE I <<< 200 Switching to Binary mode. >>> PBSZ 0 <<< 200 PBSZ set to 0. >>> PROT P <<< 200 PROT now Private. >>> PASV <<< 227 Entering Passive Mode (10,10,66,188,230,192). --- Host (10.10.66.188) Port (59072) <<+ 214 The ALLO command is supported. >>> ALLO 25 <<< 202 ALLO command ignored. >>> STOR testfile.txt <<< 150 Ok to send data. <<< 226 Transfer complete. >>> QUIT <<< 221 Goodbye.
Subject: RE: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options
Date: Thu, 15 Jan 2015 01:58:01 +0000
To: "bug-Net-FTPSSL [...] rt.cpan.org" <bug-Net-FTPSSL [...] rt.cpan.org>
From: Curtis Leach <cleach [...] Caesars.com>
Thank you for the log file and the more detailed explanation on why it failed. It strongly suggests a bug in your code or in IO-Socket-SSL if it's not including that value as part of the context in _SSL_arguments. But to prove it, I'm including another beta for you to look at. The main change is to dump the contents of SSL_Client_Certificate to the log file as well as your commented changes below. But I did see one thing when looking at the log file. Why didn't you set "SSL_use_cert=>1" in your SSL_Client_Certificate hash? You are using certificates? That might be the root cause of your issue if you are. Give it a try & let me know. I'm very interested if it fixes your problem. But I do have a couple of quick questions/comments? Not all are related to your problem. 1) Is there a reason you're using OverrideHELP It's not necessary for most servers. But if you are going to enable everything use OverrideHELP=>1 instead of OverrideHELP=>\@help_list. It looks like you manually did a HELP on some system and hard coded them into your program. You only need this option if the module bombs if you leave it out or if your server doesn't support the HELP command. It's a work around option. 2) You are not providing any values, so drop OverridePASV & Port. In both cases here these options are being ignored & default values are being used. 3) useSSL => 0 isn't needed. That's the default behavior. To use SSL do useSSL=>1, otherwise TLS is always used without this option. So a good constructor based on your logs should be: My $ftps = Net::FTPSSL->new('testserver.testdomain.com', Debug=>1, DebugLogFile=>'FTPSSL-debug.txt', Encryption=>IMP_CRYPT, Croak=>1, SSL_Client_Certificate=>\%options, OverrideHELP=>1); Drop OverrideHELLP entirely if it's not needed. Curtis Show quoted text
-----Original Message----- From: Chris Thompson via RT [mailto:bug-Net-FTPSSL@rt.cpan.org] Sent: Wednesday, January 14, 2015 1:38 PM Subject: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options Queue: Net-FTPSSL Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=101388 > Thank you for your patch. In the patch I sent, I was checking for $ssl_args{SSL_verifycn_name}, etc. In the patch that you sent, you changed it to ${*$obj}{SSL_verifycn_name}, etc. But those options are not directly within the main object, they are inside a hash called _SSL_arguments. So, if you change: ${*$obj}{SSL_verifycn_name} to ${*$obj}{_SSL_arguments}{SSL_verifycn_name} and so on, then it would almost work. The only reason it doesn't quite work is that the value for SSL_verifycn_scheme isn't going into the _SSL_arguments hash for some reason. All the other options are there, but SSL_verifycn_scheme is missing, and I don't know why. Without that option changed from the default, then the strict name checking simply doesn't happen. That's why I pulled the values from the $ssl_args instead, just to pull that missing value for SSL_verifycn_scheme. Since the PASV command could potentially return a different IP address for the data channel, then not having the SSL_verifycn_scheme option carried over to the data channel means that the strict name checking wouldn't happen on that other IP address. I've attached what the screen looks like when I turn on debugging and transfer a file (after inserting the {_SSL_arguments} hash to your patch as I mentioned above). The transfer actually happens, but the SSL_verifycn_scheme is not passed to the data channel, because it's not present in the _SSL_arguments hash for some reason. If I leave the code as it was originally when I sent my patch, then you can see the SSL_verifycn_scheme option within the myContext hash, so I know it's being used for the data channel when looking under $ssl_args instead. Please let me know if you have any questions or need anything else. Thanks, Chris Thompson ----- On Wed Jan 14 14:59:19 2015, cleach@Caesars.com wrote:
> Here it is again. The latest beta > > Curtis

Message body is not shown because sender requested not to inline it.

Subject: RE: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options
Date: Thu, 15 Jan 2015 01:59:01 +0000
To: "bug-Net-FTPSSL [...] rt.cpan.org" <bug-Net-FTPSSL [...] rt.cpan.org>
From: Curtis Leach <cleach [...] Caesars.com>
Sorry, I included the wrong beta last time. It had a bug in it. Show quoted text
-----Original Message----- From: Curtis Leach Sent: Wednesday, January 14, 2015 5:58 PM To: 'bug-Net-FTPSSL@rt.cpan.org' Subject: RE: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options Thank you for the log file and the more detailed explanation on why it failed. It strongly suggests a bug in your code or in IO-Socket-SSL if it's not including that value as part of the context in _SSL_arguments. But to prove it, I'm including another beta for you to look at. The main change is to dump the contents of SSL_Client_Certificate to the log file as well as your commented changes below. But I did see one thing when looking at the log file. Why didn't you set "SSL_use_cert=>1" in your SSL_Client_Certificate hash? You are using certificates? That might be the root cause of your issue if you are. Give it a try & let me know. I'm very interested if it fixes your problem. But I do have a couple of quick questions/comments? Not all are related to your problem. 1) Is there a reason you're using OverrideHELP It's not necessary for most servers. But if you are going to enable everything use OverrideHELP=>1 instead of OverrideHELP=>\@help_list. It looks like you manually did a HELP on some system and hard coded them into your program. You only need this option if the module bombs if you leave it out or if your server doesn't support the HELP command. It's a work around option. 2) You are not providing any values, so drop OverridePASV & Port. In both cases here these options are being ignored & default values are being used. 3) useSSL => 0 isn't needed. That's the default behavior. To use SSL do useSSL=>1, otherwise TLS is always used without this option. So a good constructor based on your logs should be: My $ftps = Net::FTPSSL->new('testserver.testdomain.com', Debug=>1, DebugLogFile=>'FTPSSL-debug.txt', Encryption=>IMP_CRYPT, Croak=>1, SSL_Client_Certificate=>\%options, OverrideHELP=>1); Drop OverrideHELLP entirely if it's not needed. Curtis
-----Original Message----- From: Chris Thompson via RT [mailto:bug-Net-FTPSSL@rt.cpan.org] Sent: Wednesday, January 14, 2015 1:38 PM Subject: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options Queue: Net-FTPSSL Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=101388 > Thank you for your patch. In the patch I sent, I was checking for $ssl_args{SSL_verifycn_name}, etc. In the patch that you sent, you changed it to ${*$obj}{SSL_verifycn_name}, etc. But those options are not directly within the main object, they are inside a hash called _SSL_arguments. So, if you change: ${*$obj}{SSL_verifycn_name} to ${*$obj}{_SSL_arguments}{SSL_verifycn_name} and so on, then it would almost work. The only reason it doesn't quite work is that the value for SSL_verifycn_scheme isn't going into the _SSL_arguments hash for some reason. All the other options are there, but SSL_verifycn_scheme is missing, and I don't know why. Without that option changed from the default, then the strict name checking simply doesn't happen. That's why I pulled the values from the $ssl_args instead, just to pull that missing value for SSL_verifycn_scheme. Since the PASV command could potentially return a different IP address for the data channel, then not having the SSL_verifycn_scheme option carried over to the data channel means that the strict name checking wouldn't happen on that other IP address. I've attached what the screen looks like when I turn on debugging and transfer a file (after inserting the {_SSL_arguments} hash to your patch as I mentioned above). The transfer actually happens, but the SSL_verifycn_scheme is not passed to the data channel, because it's not present in the _SSL_arguments hash for some reason. If I leave the code as it was originally when I sent my patch, then you can see the SSL_verifycn_scheme option within the myContext hash, so I know it's being used for the data channel when looking under $ssl_args instead. Please let me know if you have any questions or need anything else. Thanks, Chris Thompson ----- On Wed Jan 14 14:59:19 2015, cleach@Caesars.com wrote:
> Here it is again. The latest beta > > Curtis

Message body is not shown because sender requested not to inline it.

Subject: RE: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options
Date: Wed, 21 Jan 2015 23:11:22 +0000
To: "bug-Net-FTPSSL [...] rt.cpan.org" <bug-Net-FTPSSL [...] rt.cpan.org>
From: Curtis Leach <cleach [...] Caesars.com>
Chris, Will you be trying out the attached patch & returning the logs to me? I'd just like to verify I fully understood your problem (via the logs) and that the patch resolved any issues you were encountering. Curtis Show quoted text
-----Original Message----- From: Curtis Leach Sent: Wednesday, January 14, 2015 5:59 PM To: 'bug-Net-FTPSSL@rt.cpan.org' Subject: RE: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options Sorry, I included the wrong beta last time. It had a bug in it.
-----Original Message----- From: Curtis Leach Sent: Wednesday, January 14, 2015 5:58 PM To: 'bug-Net-FTPSSL@rt.cpan.org' Subject: RE: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options Thank you for the log file and the more detailed explanation on why it failed. It strongly suggests a bug in your code or in IO-Socket-SSL if it's not including that value as part of the context in _SSL_arguments. But to prove it, I'm including another beta for you to look at. The main change is to dump the contents of SSL_Client_Certificate to the log file as well as your commented changes below. But I did see one thing when looking at the log file. Why didn't you set "SSL_use_cert=>1" in your SSL_Client_Certificate hash? You are using certificates? That might be the root cause of your issue if you are. Give it a try & let me know. I'm very interested if it fixes your problem. But I do have a couple of quick questions/comments? Not all are related to your problem. 1) Is there a reason you're using OverrideHELP It's not necessary for most servers. But if you are going to enable everything use OverrideHELP=>1 instead of OverrideHELP=>\@help_list. It looks like you manually did a HELP on some system and hard coded them into your program. You only need this option if the module bombs if you leave it out or if your server doesn't support the HELP command. It's a work around option. 2) You are not providing any values, so drop OverridePASV & Port. In both cases here these options are being ignored & default values are being used. 3) useSSL => 0 isn't needed. That's the default behavior. To use SSL do useSSL=>1, otherwise TLS is always used without this option. So a good constructor based on your logs should be: My $ftps = Net::FTPSSL->new('testserver.testdomain.com', Debug=>1, DebugLogFile=>'FTPSSL-debug.txt', Encryption=>IMP_CRYPT, Croak=>1, SSL_Client_Certificate=>\%options, OverrideHELP=>1); Drop OverrideHELLP entirely if it's not needed. Curtis
-----Original Message----- From: Chris Thompson via RT [mailto:bug-Net-FTPSSL@rt.cpan.org] Sent: Wednesday, January 14, 2015 1:38 PM Subject: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options Queue: Net-FTPSSL Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=101388 > Thank you for your patch. In the patch I sent, I was checking for $ssl_args{SSL_verifycn_name}, etc. In the patch that you sent, you changed it to ${*$obj}{SSL_verifycn_name}, etc. But those options are not directly within the main object, they are inside a hash called _SSL_arguments. So, if you change: ${*$obj}{SSL_verifycn_name} to ${*$obj}{_SSL_arguments}{SSL_verifycn_name} and so on, then it would almost work. The only reason it doesn't quite work is that the value for SSL_verifycn_scheme isn't going into the _SSL_arguments hash for some reason. All the other options are there, but SSL_verifycn_scheme is missing, and I don't know why. Without that option changed from the default, then the strict name checking simply doesn't happen. That's why I pulled the values from the $ssl_args instead, just to pull that missing value for SSL_verifycn_scheme. Since the PASV command could potentially return a different IP address for the data channel, then not having the SSL_verifycn_scheme option carried over to the data channel means that the strict name checking wouldn't happen on that other IP address. I've attached what the screen looks like when I turn on debugging and transfer a file (after inserting the {_SSL_arguments} hash to your patch as I mentioned above). The transfer actually happens, but the SSL_verifycn_scheme is not passed to the data channel, because it's not present in the _SSL_arguments hash for some reason. If I leave the code as it was originally when I sent my patch, then you can see the SSL_verifycn_scheme option within the myContext hash, so I know it's being used for the data channel when looking under $ssl_args instead. Please let me know if you have any questions or need anything else. Thanks, Chris Thompson ----- On Wed Jan 14 14:59:19 2015, cleach@Caesars.com wrote:
> Here it is again. The latest beta > > Curtis

Message body is not shown because sender requested not to inline it.

From: arthurdent99 [...] hotmail.com
Hello, The patch that you sent works fine, except that the four print_LOG() statements that you added cause the file transfer to crash. I noticed the same thing when I was experimenting with my initial patch. The _debug_print_hash() that you added works fine, but apparently you can't use print_LOG() in that section of the code without causing it to fail. So I just commented out those four lines (lines 360, 364, 368, and 371). After I commented out those lines, the test ran fine. I'm not actually using an SSL client certificate in this case. I'm only setting the SSL_Client_Certificate option to trigger the reuse of the SSL context and to pass the options for strict hostname verification. I've attached the debug log from the test (after I commented out the four lines mentioned above). Thanks, Chris Thompson ----- On Wed Jan 21 18:11:41 2015, cleach@Caesars.com wrote: Show quoted text
> Chris, > > Will you be trying out the attached patch & returning the logs to me? > > I'd just like to verify I fully understood your problem (via the logs) > and that the patch resolved any issues you were encountering. > > Curtis
Subject: FTPSSL-debug.txt
Net-FTPSSL Version: 0.26 Perl: 5.008008 [5.8.8], OS: linux Server (port): testserver.testdomain.com (990) Keys: (Croak), (Debug), (Encryption), (SSL_Client_Certificate), (DebugLogFile) Values: (1), (1), (I), (HASH(0xa31f0b0)), (FTPSSL-debug.txt) <<< 220 (vsFTPd 2.2.2) Object HASH Details ... (SSL_Client_Certificate:options - I) SSL_ca_file ==> /etc/pki/tls/certs/ca-bundle.crt SSL_verify_mode ==> 1 SSL_verifycn_name ==> testserver.testdomain.com SSL_verifycn_scheme ==> ftp Object Net::FTPSSL Details ... (testserver.testdomain.com:990 - I) Croak ==> 1 Crypt ==> I FixGetTs ==> 0 FixPutTs ==> 0 Host ==> testserver.testdomain.com OverridePASV ==> (undef) Pret ==> 0 Timeout ==> 120 _SSL_arguments ==> HASH(0xa4013d8) -- PeerAddr ===> 10.10.66.188 -- PeerPort ===> 990 -- Proto ===> tcp -- SSL_ca_file ===> /etc/pki/tls/certs/ca-bundle.crt -- SSL_check_crl ===> 0 -- SSL_cipher_list ===> ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES128-SHA256 ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES256-SHA384 ECDHE-ECDSA-AES128-SHA ECDHE-ECDSA-AES256-SHA ECDHE-RSA-AES128-SHA256 ECDHE-RSA-AES128-SHA ECDHE-RSA-AES256-SHA DHE-DSS-AES128-SHA256 DHE-DSS-AES128-SHA DHE-DSS-AES256-SHA256 DHE-DSS-AES256-SHA AES128-SHA256 AES128-SHA AES256-SHA256 AES256-SHA EDH-DSS-DES-CBC3-SHA DES-CBC3-SHA RC4-SHA !EXP !LOW !eNULL !aNULL !DES !MD5 !PSK !SRP -- SSL_server ===> 0 -- SSL_use_cert ===> 0 -- SSL_verify_callback ===> CODE(0xa4013fc) -- SSL_verify_mode ===> 1 -- SSL_verifycn_name ===> testserver.testdomain.com -- SSL_version ===> TLSv1 _SSL_ctx ==> IO::Socket::SSL::SSL_Context=HASH(0xa4014bc) -- context ===> 171939576 -- ocsp_mode ===> 16 -- verify_mode ===> 1 -- verify_name_ref ===> SCALAR(0xa1b8808) _SSL_fileno ==> 5 _SSL_ioclass_upgraded ==> IO::Socket::INET _SSL_last_err ==> SSL wants a read first _SSL_object ==> 172428400 _SSL_opened ==> 1 buf_size ==> 10240 data_prot ==> P dcsc_mode ==> 1 debug ==> 2 debug_extra ==> 0 ftpssl_filehandle ==> GLOB(0xa3ec764) io_socket_domain ==> 2 io_socket_proto ==> 6 io_socket_timeout ==> 120 io_socket_type ==> 1 last_ftp_msg ==> 220 (vsFTPd 2.2.2) myContext ==> HASH(0xa2d0d64) -- SSL_ca_file ===> /etc/pki/tls/certs/ca-bundle.crt -- SSL_reuse_ctx ===> IO::Socket::SSL::SSL_Context=HASH(0xa4014bc) -- context ----> 171939576 -- ocsp_mode ----> 16 -- verify_mode ----> 1 -- verify_name_ref ----> SCALAR(0xa1b8808) -- SSL_verifycn_name ===> testserver.testdomain.com -- SSL_verifycn_scheme ===> ftp mySocketOpts ==> HASH(0xa2d0860) -- PeerAddr ===> testserver.testdomain.com -- PeerPort ===> 990 -- Proto ===> tcp -- Timeout ===> 120 trace ==> 0 type ==> A >>> USER +++++++ <<< 331 Please specify the password. >>> PASS ******* <<< 230 Login successful. >>> HELP <<< 214-The following commands are recognized. <<< ABOR ACCT ALLO APPE CDUP CWD DELE EPRT EPSV FEAT HELP LIST MDTM MKD <<< MODE NLST NOOP OPTS PASS PASV PORT PWD QUIT REIN REST RETR RMD RNFR <<< RNTO SITE SIZE SMNT STAT STOR STOU STRU SYST TYPE USER XCUP XCWD XMKD <<< XPWD XRMD <<< 214 Help OK. >>> FEAT <<< 211-Features: <<< AUTH SSL <<< AUTH TLS <<< EPRT <<< EPSV <<< MDTM <<< PASV <<< PBSZ <<< PROT <<< REST STREAM <<< SIZE <<< TVFS <<< UTF8 <<< 211 End <<+ 214 The HELP command is supported. >>> TYPE I <<< 200 Switching to Binary mode. >>> PBSZ 0 <<< 200 PBSZ set to 0. >>> PROT P <<< 200 PROT now Private. >>> PASV <<< 227 Entering Passive Mode (10,10,66,188,188,152). --- Host (10.10.66.188) Port (48280) <<+ 214 The ALLO command is supported. >>> ALLO 25 <<< 202 ALLO command ignored. >>> STOR testfile.txt <<< 150 Ok to send data. <<< 226 Transfer complete. >>> QUIT <<< 221 Goodbye.
Subject: RE: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options
Date: Thu, 22 Jan 2015 19:34:34 +0000
To: "bug-Net-FTPSSL [...] rt.cpan.org" <bug-Net-FTPSSL [...] rt.cpan.org>
From: Curtis Leach <cleach [...] Caesars.com>
Thank you for the logs & the clarification on the issue. The log helped a lot. I found the typos. It was missing the leading underscore in the function name. $obj->print_LOG (...) Should have been $obj->_print_LOG(...) But you are right about calls in new(), calling member functions don't always work. It doesn't work until the command channel is opened & it gets blessed to be an FTPSSL object. If you need to call one before that you have to call it in a different way. I have a couple more tweaks to make for the logging, but I should have an official release uploaded by early next week. But I'm going to have to re-read the IO-Socket-SSL pod text more closely before I do. It's changed quite a bit since I last reviewed things. And it's probably time to closely review it again for new surprises. Thank you for your help in this. Curtis Show quoted text
-----Original Message----- From: Chris Thompson via RT [mailto:bug-Net-FTPSSL@rt.cpan.org] Sent: Thursday, January 22, 2015 8:21 AM Subject: [rt.cpan.org #101388] SSL_reuse_ctx is missing certain options Queue: Net-FTPSSL Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=101388 > Hello, The patch that you sent works fine, except that the four print_LOG() statements that you added cause the file transfer to crash. I noticed the same thing when I was experimenting with my initial patch. The _debug_print_hash() that you added works fine, but apparently you can't use print_LOG() in that section of the code without causing it to fail. So I just commented out those four lines (lines 360, 364, 368, and 371). After I commented out those lines, the test ran fine. I'm not actually using an SSL client certificate in this case. I'm only setting the SSL_Client_Certificate option to trigger the reuse of the SSL context and to pass the options for strict hostname verification. I've attached the debug log from the test (after I commented out the four lines mentioned above). Thanks, Chris Thompson ----- On Wed Jan 21 18:11:41 2015, cleach@Caesars.com wrote:
> Chris, > > Will you be trying out the attached patch & returning the logs to me? > > I'd just like to verify I fully understood your problem (via the logs) > and that the patch resolved any issues you were encountering. > > Curtis
Chris, Thank you for your help outside the chain of this ticket. I've finally uploaded the official release. There were just minor changes to the beta I provided earlier. So I'm closing this ticket. Thanks for using my module. Curtis