Bug #73532 for Net-SSLeay: using Net::SSLeay in threads

Wed Dec 28 04:06:18 2011 info [...] maximka.de - Ticket created

Subject:

using Net::SSLeay in threads

Hi, unfortunately is using of Net::SSLeay for HTTPS requests in perl-threads impossible. The on ticket attached script produce in all test environments, such as debian lenny and squeeze or ubuntu 10.4, following exception: *** glibc detected *** perl: double free or corruption (fasttop): 0xb58d0f80 *** ======= Backtrace: ========= /lib/tls/i686/cmov/libc.so.6(+0x6b591)[0xe5b591] /lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0xe5cde8] /lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0xe5fecd] /lib/i686/cmov/libcrypto.so.0.9.8(CRYPTO_free+0x3a)[0x31a03a] /lib/i686/cmov/libcrypto.so.0.9.8(OBJ_NAME_add+0x9d)[0x31d7cd] /lib/i686/cmov/libcrypto.so.0.9.8(EVP_add_cipher+0x5a)[0x38f71a] /lib/i686/cmov/libssl.so.0.9.8(SSL_library_init+0x26)[0x14d0c6] /usr/lib/perl5/auto/Net/SSLeay/SSLeay.so(XS_Net__SSLeay_library_init+0xe0)[0x1fe070] perl(Perl_pp_entersub+0x533)[0x80d5af3] perl(Perl_runops_standard+0x18)[0x80d3ee8] perl(Perl_call_sv+0x42a)[0x807beaa] /usr/lib/perl/5.10/auto/threads/threads.so(+0x60b2)[0xb130b2] /lib/tls/i686/cmov/libpthread.so.0(+0x596e)[0x18e96e] /lib/tls/i686/cmov/libc.so.6(clone+0x5e)[0xebda4e] Best regards, palik

Subject:

threads_net_ssleay-2.pl

#!/usr/bin/perl use v5.10; use strict; use warnings; use Net::SSLeay; use threads; #$Net::SSLeay::trace = 2; my $i = 1; while (1) { threads->new('create_thread') for (1 .. 3); foreach (threads->list) { $_->join; } say "loop(", $i++, ") ", '-' x 20; } ## end while (1) sub create_thread { my $tid = threads->tid; threads->set_thread_exit_only(1); print "\tcreate_thread($tid)"; sleep(rand(3)); my ($page, $response, %reply_headers) = Net::SSLeay::get_https('localhost', 443, '/'); say join " ", "\tresp-code:", $response, "length:", length($page); threads->exit(0); } ## end sub create_thread

Fri Jan 13 16:42:30 2012 MIKEM [...] cpan.org - Correspondence added

Hi, initial tests here show this working OK, so prob some dependency on perl or os version. Can you please tell me exact versions of: os perl openssl net-ssleay that you test with? Cheers.

Fri Jan 13 16:42:32 2012 The RT System itself - Status changed from 'new' to 'open'

Fri Jan 13 16:42:32 2012 MIKEM [...] cpan.org - Given to MIKEM

Mon Jan 16 12:26:34 2012 info [...] maximka.de - Correspondence added

Hi, thanks for your response. It takes sometimes the hours before the script produce the execution. But as I said in all fallowing environments the script died: Debian GNU/Linux 6.0 perl 5.10.1 openssl 0.9.8o-4squeeze4 Net::SSLeay 1.36 -- Debian GNU/Linux 5.0 perl 5.10.1 openssl 0.9.8g-15+lenny14 Net::SSLeay 1.35 -- Ubuntu 10.04.3 LTS perl 5.10.1 openssl 0.9.8k-7ubuntu8.6 Net::SSLeay 1.35 regards, palik On Fri Jan 13 16:42:30 2012, MIKEM wrote: Show quoted text

> Hi, > initial tests here show this working OK, so prob some dependency on > perl or os version. > > Can you please tell me exact versions of: > os > perl > openssl > net-ssleay > > that you test with? > > Cheers.

Tue Jan 17 03:36:07 2012 kmx [...] cpan.org - Correspondence added

According to http://www.openssl.org/docs/crypto/threads.html to be using openssl in thread-safe manners we need to define some special locking related functions.

See enclosed patch with my idea how to implement it - BEWARE: it is just an idea which still crashed couple of times during my testing - unfortunately not easily reproducible.

Another issue might be related to static data in XS code. According http://perldoc.perl.org/perlxs.html it might be a good idea to armour them with MY_CXT* macros (see http://perldoc.perl.org/perlxs.html).

--
kmx

Subject:

net-ssleay-thead-safety-idea.diff

Index: SSLeay.xs =================================================================== --- SSLeay.xs (revision 282) +++ SSLeay.xs (working copy) @@ -85,6 +85,92 @@ typedef int perl_filehandle_t; +/* ============= thread-safely calling openssl lib ============== */ +/* more info: http://www.openssl.org/docs/crypto/threads.html */ + +#ifdef USE_ITHREADS + +#if defined(WIN32) + +static HANDLE *lock_cs; +static DWORD creator_thread_id; + +static void openssl_locking_function(int mode, int type, const char *file, int line) +{ + if (mode & CRYPTO_LOCK) + WaitForSingleObject(lock_cs[type],INFINITE); + else + ReleaseMutex(lock_cs[type]); +} + +static void openssl_threads_init() +{ + int i; + lock_cs=malloc(CRYPTO_num_locks() * sizeof(HANDLE)); + creator_thread_id = GetCurrentThreadId(); + for (i=0; i<CRYPTO_num_locks(); i++) lock_cs[i]=CreateMutex(NULL,FALSE,NULL); + CRYPTO_set_locking_callback((void (*)(int,int,const char *,int))openssl_locking_function); + /* no need for threadid_func() on Win32 - see openssl docuemtation */ +} + +static void openssl_threads_cleanup() +{ + int i; + CRYPTO_set_locking_callback(NULL); + if (lock_cs && creator_thread_id == GetCurrentThreadId()) { + for (i=0; i<CRYPTO_num_locks(); i++) CloseHandle(lock_cs[i]); + free(lock_cs); + } +} + +#elif defined(PTHREADS) + +static void openssl_locking_function(int mode, int type, const char *file, int line) +{ + if (mode & CRYPTO_LOCK) + pthread_mutex_lock(&(lock_cs[type])); + else + pthread_mutex_unlock(&(lock_cs[type])); +} + +static unsigned long openssl_threadid_func() +{ + return (unsigned long)pthread_self(); +} + +static void openssl_threads_init() +{ + int i; + lock_cs=malloc(CRYPTO_num_locks() * sizeof(pthread_mutex_t)); + for (i=0; i<CRYPTO_num_locks(); i++) pthread_mutex_init(&(lock_cs[i]),NULL); + CRYPTO_set_locking_callback((void (*)(int,int,const char *,int))openssl_locking_function); + CRYPTO_set_id_callback((unsigned long (*)())openssl_threadid_func); +} + +static void openssl_threads_cleanup() +{ + int i; + CRYPTO_set_locking_callback(NULL); + CRYPTO_set_id_callback(NULL); + for (i=0; i<CRYPTO_num_locks(); i++) pthread_mutex_destroy(&(dMY_CXTlock_cs[i])); + free(lock_cs); +} + +#else + +static void openssl_threads_init() +{ + /* "Warning: unknown threads framework, Net::SSLeay cannot be used thread-safely!\n" */ +} + +static void openssl_threads_cleanup() +{ +} + +#endif + +#endif + /* ============= callback stuff ============== */ static HV* ssleay_ctx_verify_callbacks = (HV*)NULL; @@ -649,6 +735,22 @@ PROTOTYPES: ENABLE +BOOT: +{ +#ifdef USE_ITHREADS + openssl_threads_init(); +#endif +} + +void +END(...) +CODE: +{ +#ifdef USE_ITHREADS + openssl_threads_cleanup(); +#endif +} + double constant(name) char * name

Tue Jan 17 03:37:01 2012 kmx [...] cpan.org - Cc KMX added

Tue Jan 17 11:27:56 2012 kmx [...] cpan.org - Correspondence added

I have prepared more intensive crash-test. See enclosed tarball (instructions in readme.txt).

Should crash within no more than 1 minute.

--
kmx

Subject:

net-ssleay-thread-crash.tar.gz

Download net-ssleay-thread-crash.tar.gz
application/x-tar 3.7k

Message body not shown because it is not plain text.

Tue Jan 17 22:26:59 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 18 Jan 2012 13:26:31 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, thanks for that. I can reproduce this bug now. Working..... Cheers. On Tuesday, January 17, 2012 11:27:58 AM kmx via RT wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > I have prepared more intensive crash-test. See enclosed tarball > (instructions in readme.txt). > > Should crash within no more than 1 minute. > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Wed Jan 18 01:31:36 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 18 Jan 2012 16:31:05 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, thanks for the crash-test. It appears this problem is due to the SSL library being reinitilised multiple times in the threads. Net::SSLeay::get_https reinitialises the library by calling SSLeay_add_ssl_algorithms (which is really SSL_library_init), however, SSL_library_init is not reentrant, according to http://www.openssl.org/docs/ssl/SSL_library_init.html and its seems that overlapping calls cause the double free. If I arrange for the library to be initiliased exactly once, your crash-test runs forever (or at least until the server side runs out of memory: it leaks badly) Seems that get_https was written long ago before threading was an issue, and making substantial changes to it now would break existing code. Its not clear to me what is the best way to fix this, though: each thread cant see the perl variables in the other threads, so we cant use a perl flag to indicate that a thread has already initilised the SSL library. Seems that the SSL library would have to be initialised once in the _calling_ code before threading starts, but this is going to need a change to calling code, as well as get_https. I cant see a way to contain the fix within SSLeay.pm :-( Cheers. On Tuesday, January 17, 2012 10:27:00 PM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > Hi, > > thanks for that. > I can reproduce this bug now. > Working..... > > Cheers. > > On Tuesday, January 17, 2012 11:27:58 AM kmx via RT wrote:

> > Queue: Net-SSLeay > > > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > > > I have prepared more intensive crash-test. See enclosed tarball > > (instructions in readme.txt). > > > > Should crash within no more than 1 minute. > > > > -- > > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Wed Jan 18 05:02:06 2012 info [...] maximka.de - Correspondence added

Hi, Show quoted text

> Seems that get_https was written long ago before threading was

My script use get_https to simplify testing. But the sslcat and post_https methods produce same exception. regards, palik

Wed Jan 18 06:27:17 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 18 Jan 2012 21:26:36 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

On Wednesday, January 18, 2012 05:02:07 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > Hi, >

> > Seems that get_https was written long ago before threading was

> > My script use get_https to simplify testing. But the sslcat and post_https > methods produce same exception.

Yes, both of those have the same problem for the same reason. Cheers. Show quoted text

> > regards, > palik

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Wed Jan 18 13:46:24 2012 kmx [...] volny.cz - Correspondence added

CC:	info [...] maximka.de
Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 18 Jan 2012 19:46:10 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> Net::SSLeay::get_https reinitialises the library by calling > SSLeay_add_ssl_algorithms (which is really SSL_library_init), however, > SSL_library_init is not reentrant, according to > http://www.openssl.org/docs/ssl/SSL_library_init.html > and its seems that overlapping calls cause the double free. > > If I arrange for the library to be initiliased exactly once, your crash-test > runs forever (or at least until the server side runs out of memory: it leaks > badly) >

Do you have a guess whether it is my code or Net::SSLeay leaking? The thing is that I am more interested in multi-threaded use of Net::SSLeay on server side. Show quoted text

> Seems that get_https was written long ago before threading was an issue, and > making substantial changes to it now would break existing code. > > Its not clear to me what is the best way to fix this, though: each thread cant > see the perl variables in the other threads, so we cant use a perl flag to > indicate that a thread has already initilised the SSL library. >

To share a variable between threads perhaps SvSHARE might come handy (see http://perldoc.perl.org/perlapi.html#Magical-Functions) but it will probably also require some kind of locking. Show quoted text

> Seems that the SSL library would have to be initialised once in the _calling_ > code before threading starts, but this is going to need a change to calling > code, as well as get_https. I cant see a way to contain the fix within > SSLeay.pm :-( >

What about to use BOOT: section of SSLeay.xs like this: 1/ put into BOOT: load_error_strings(); SSLeay_add_ssl_algorithms(); (which - I hope - means that this initialization will done exactly once when you "use Net::SSLeay") 2/ remove all usage of SSLeay_add_ssl_algorithms and load_error_strings from *.pm 3/ in SSLeay.xs replace SSL_library_init and load_error_strings with empty subs or deprecation warning -- kmx

Wed Jan 18 17:52:24 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 19 Jan 2012 08:51:52 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, On Wednesday, January 18, 2012 01:46:26 PM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >

> > Net::SSLeay::get_https reinitialises the library by calling > > SSLeay_add_ssl_algorithms (which is really SSL_library_init), however, > > SSL_library_init is not reentrant, according to > > http://www.openssl.org/docs/ssl/SSL_library_init.html > > and its seems that overlapping calls cause the double free. > > > > If I arrange for the library to be initiliased exactly once, your > > crash-test runs forever (or at least until the server side runs out of > > memory: it leaks badly)

> > Do you have a guess whether it is my code or Net::SSLeay leaking? The > thing is that I am more interested in multi-threaded use of Net::SSLeay > on server side.

I have not looked. But I can tell you that I use net-ssleay a lot as a server without leaks. However I dont use the Net::SSLeay::set_fd interface. Show quoted text

>

> > Seems that get_https was written long ago before threading was an issue, > > and making substantial changes to it now would break existing code. > > > > Its not clear to me what is the best way to fix this, though: each > > thread cant see the perl variables in the other threads, so we cant use > > a perl flag to indicate that a thread has already initilised the SSL > > library.

> > To share a variable between threads perhaps SvSHARE might come handy > (see http://perldoc.perl.org/perlapi.html#Magical-Functions) but it will > probably also require some kind of locking. >

> > Seems that the SSL library would have to be initialised once in the > > _calling_ code before threading starts, but this is going to need a > > change to calling code, as well as get_https. I cant see a way to > > contain the fix within SSLeay.pm :-(

> > What about to use BOOT: section of SSLeay.xs like this: > > 1/ put into BOOT: > load_error_strings(); > SSLeay_add_ssl_algorithms(); > (which - I hope - means that this initialization will done exactly once > when you "use Net::SSLeay") > > 2/ remove all usage of SSLeay_add_ssl_algorithms and load_error_strings > from *.pm > > 3/ in SSLeay.xs replace SSL_library_init and load_error_strings with > empty subs or deprecation warning

Except that some people need to initialise in a different way, and at different times. My preferred option now is to change SSLeay.pm so: 1. The initilaisation code that the SSLeay.pm functions use is moved to a sub, and the sub uses a flag to only initialise once. This will allow existing non- threaded code that uses https_cat and sslcat to continue to run unchanged 2. Require code that uses https_cat and sslcat in multiple threads to call that initialisation function before threading starts. Code that does not use https_cat and sslcat would continue to initialise the library in their current way unchanged, but they could use the new init function if they wanted. I think this means that existing code will continue to run, and the only code that would have to change is threaded code that uses https_cat and sslcat (like yours) which does not work correctly anyway. Views? Cheers. Show quoted text

> > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Thu Jan 19 04:41:48 2012 kmx [...] volny.cz - Correspondence added

CC:	info [...] maximka.de
Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 19 Jan 2012 10:41:32 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Hi, Show quoted text

>> Do you have a guess whether it is my code or Net::SSLeay leaking? The >> thing is that I am more interested in multi-threaded use of Net::SSLeay >> on server side. >>

> I have not looked. But I can tell you that I use net-ssleay a lot as a server > without leaks. However I dont use the Net::SSLeay::set_fd interface. >

As I have a tcp dispatcher in a separate thread I have to pass somehow the accepted socket between threads. As far as I have googled it is not completely safe to pass IO::Socket::INET object, therefore I am passing fileno and need to call set_fd. If we manage to make Net::SSLeay thread-safe enough I can polish a bit my crash test scripts and turn them into <Net-SSLeay Show quoted text

tarball>/examples/ssl-http-server-multi-threaded.pl + <Net-SSLeay tarball>/examples/ssl-http-client-multi-threaded.pl

Show quoted text

> My preferred option now is to change SSLeay.pm so: > > 1. The initilaisation code that the SSLeay.pm functions use is moved to a sub, > and the sub uses a flag to only initialise once. This will allow existing non- > threaded code that uses https_cat and sslcat to continue to run unchanged > > 2. Require code that uses https_cat and sslcat in multiple threads to call > that initialisation function before threading starts. > > Code that does not use https_cat and sslcat would continue to initialise the > library in their current way unchanged, but they could use the new init > function if they wanted. > > I think this means that existing code will continue to run, and the only code > that would have to change is threaded code that uses https_cat and sslcat > (like yours) which does not work correctly anyway. > > Views? >

I agree, your suggestion is more back-compat-friendly. As mentioned before I prefer to do proper locking to make it as thread-safe as possible - which means something like this: [... near the start of SSLeay.xs ...] /* ============= thread-safety related stuff ============== */ #define MY_CXT_KEY "Net::SSLeay::_guts" XS_VERSION typedef struct { int initialized; #ifdef USE_ITHREADS perl_mutex init_mutex; #endif } my_cxt_t; START_MY_CXT; [... later ...] BOOT: MY_CXT_INIT; MY_CXT.initialized = 0; #ifdef USE_ITHREADS MUTEX_INIT(&MY_CXT.init_mutex); #endif void END(...) CODE: dMY_CXT; #ifdef USE_ITHREADS MUTEX_DESTROY(&MY_CXT.init_mutex); #endif void CLONE(...) CODE: MY_CXT_CLONE; [... new SSL_library_init() ...] int SSL_library_init() ALIAS: SSLeay_add_ssl_algorithms = 1 OpenSSL_add_ssl_algorithms = 2 add_ssl_algorithms = 3 CODE: dMY_CXT; #ifdef USE_ITHREADS MUTEX_LOCK(&MY_CXT.init_mutex); #endif if (!MY_CXT.initialized) { SSL_library_init(); #ifdef USE_ITHREADS /* maybe do also some other thread-safety init things here: CRYPTO_set_locking_callback & co. */ #endif MY_CXT.initialized = 1; } #ifdef USE_ITHREADS MUTEX_UNLOCK(&MY_CXT.init_mutex); #endif TODO: 1/ I am not sure what exactly should be done in CLONE() and END() functions 1/ might it be good for something/someone to have a function like SSL_libraty_init_status() - returning current MY_CXT.initialized value ? 3/ would you agree to add also things related to CRYPTO_set_locking_callback ? see http://www.openssl.org/docs/crypto/threads.html or multi-threaded example http://www.opensource.apple.com/source/OpenSSL/OpenSSL-10/openssl/crypto/threads/mttest.c (or check mttest.c from original openssl tarball) -- kmx

Thu Jan 19 05:16:43 2012 info [...] maximka.de - Correspondence added

Hi, Show quoted text

> As far as I have googled it is not > completely safe to pass IO::Socket::INET object, therefore I am > passing

I'm tried to use IO::SOCKET::SSL, wich ist inherits von IO::Socket::INET. It's not thread safe. But Socket::Class::SSL works well with threads. regards, palik

Thu Jan 19 05:45:33 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 19 Jan 2012 11:45:24 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> I'm tried to use IO::SOCKET::SSL, wich ist inherits von IO::Socket::INET. It's not thread safe. > But Socket::Class::SSL works well with threads. >

Thanks a lot for pointig to Socket::Class::SSL Which makes me think about a scenario when we have two different modules using openssl loaded in the same multithreaded perl application. e.g. 1/ NetSSLeay 2/ Socket::Class::SSL Both of them will at some point call non-reentrant SSL_library_init() function (either from the main or another thread) - I have a feeling that some strange things can happen. -- kmx

Thu Jan 19 08:24:21 2012 info [...] maximka.de - Correspondence added

Show quoted text

> Which makes me think about a scenario when we have two different > modules > using openssl loaded in the same multithreaded perl application. > > e.g. > 1/ NetSSLeay > 2/ Socket::Class::SSL

I couldn't see any reason to use both of them in a same application at all, but it's probable. There are many ifdef USE_ITHREADS blocks in -- Socket-Class-2.256$ grep -r USE_ITHREADS * Class.c:#ifdef USE_ITHREADS ... -- I added "use Net::SSLeay;" in a Socket::Class::SSL based test applikation. It works. regards, palik

Thu Jan 19 10:07:10 2012 kmx [...] volny.cz - Correspondence added

CC:	kmx [...] cpan.org
Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 19 Jan 2012 16:07:00 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> I couldn't see any reason to use both of them in a same application at all, but it's probable. >

For example I not only need SSLized sockets but also to do some X509 related things (but yes, in the end there will be some workaround) Show quoted text

> There are many ifdef USE_ITHREADS blocks in > -- > Socket-Class-2.256$ grep -r USE_ITHREADS * > Class.c:#ifdef USE_ITHREADS > ... > -- > > I added "use Net::SSLeay;" in a Socket::Class::SSL based test applikation. It works. >

You are right Socket::Class(::SSL) is definitely written with threads in mind but I have tried to rewrite the server side of my crash-test to Socket::Class::SSL and I am able to crash it as well (well, it is much harder, but it crashes). Things are maybe worse due to the fact that I am testing primarily on perl+Win32 platform. -- kmx

Thu Jan 19 10:21:14 2012 kmx [...] volny.cz - Correspondence added

CC:	info [...] maximka.de
Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 19 Jan 2012 16:21:04 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> I have not looked. But I can tell you that I use net-ssleay a lot as a server > without leaks. However I dont use the Net::SSLeay::set_fd interface. >

my fault, there is missing Net::SSLeay::CTX_free in sub thread_worker (httpd-multithreaded.pl) -- kmx

Thu Jan 19 10:35:08 2012 kmx [...] cpan.org - Correspondence added

My today's Net::SSLeay hacking ended up with enclosed net-ssleay-thead-safety-proposal1.diff (diff against current SVN trunk).

Crash test seems to be running fine for a very long time - testing on more-trouble-making-platform Win32.

I am still not sure whether to completely ignore openssl locking described in http://www.openssl.org/docs/crypto/threads.html

--
kmx

Thu Jan 19 10:35:55 2012 kmx [...] cpan.org - Correspondence added

now the attachment

Subject:

net-ssleay-thead-safety-proposal1.diff

Index: SSLeay.xs =================================================================== --- SSLeay.xs (revision 282) +++ SSLeay.xs (working copy) @@ -74,6 +74,18 @@ #include "constants.c" +/* ============= thread-safety related stuff ============== */ + +#define MY_CXT_KEY "Net::SSLeay::_guts" XS_VERSION + +typedef struct { + int initialized; +#ifdef USE_ITHREADS + perl_mutex init_mutex; +#endif +} my_cxt_t; +START_MY_CXT; + /* ============= typedefs to agument TYPEMAP ============== */ typedef void callback_no_ret(void); @@ -649,6 +661,28 @@ PROTOTYPES: ENABLE +BOOT: + MY_CXT_INIT; + MY_CXT.initialized = 0; +#ifdef USE_ITHREADS + MUTEX_INIT(&MY_CXT.init_mutex); +#endif + +void +END(...) +CODE: +#ifdef USE_ITHREADS + dMY_CXT; + MUTEX_DESTROY(&MY_CXT.init_mutex); +#endif + +void +CLONE(...) +CODE: +#ifdef USE_ITHREADS + MY_CXT_CLONE; +#endif + double constant(name) char * name @@ -1332,10 +1366,22 @@ int SSL_library_init() - ALIAS: - SSLeay_add_ssl_algorithms = 1 - OpenSSL_add_ssl_algorithms = 2 - add_ssl_algorithms = 3 + ALIAS: + SSLeay_add_ssl_algorithms = 1 + OpenSSL_add_ssl_algorithms = 2 + add_ssl_algorithms = 3 + CODE: + dMY_CXT; +#ifdef USE_ITHREADS + MUTEX_LOCK(&MY_CXT.init_mutex); +#endif + if (!MY_CXT.initialized) { + SSL_library_init(); + MY_CXT.initialized = 1; + } +#ifdef USE_ITHREADS + MUTEX_UNLOCK(&MY_CXT.init_mutex); +#endif void ENGINE_load_builtin_engines()

Thu Jan 19 11:14:06 2012 info [...] maximka.de - Correspondence added

On Thu Jan 19 10:35:08 2012, KMX wrote: Show quoted text

> My today's Net::SSLeay hacking ended up with enclosed > net-ssleay-thead-safety-proposal1.diff (diff against current SVN > trunk).

after applying your patch to SSLeay.xs from cpan tarball http://search.cpan.org/CPAN/authors/id/M/MI/MIKEM/Net-SSLeay-1.42.tar.gz your net-ssleay-thread-crash scripts work well on Ubuntu 10.04.3 LTS too. regards, palik

Fri Jan 20 11:26:16 2012 info [...] maximka.de - Correspondence added

Hi, Show quoted text

> Crash test seems to be running fine for a very long time - testing on > more-trouble-making-platform Win32.

It works debian-squeeze without crashing too. An other test script, each thread execute two https requests first with Net::SSLeay::sslcat and second with Socket::Class::SSL::write/read didn't crashed too. But there is a bad thing: memory usage growth continuous and fast. regards, palik

Sat Jan 21 18:34:23 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Sun, 22 Jan 2012 09:33:59 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi all, I have now implemented the preferred solution as described below. It is now available in CVS. In order to use this fix in a multi threaded environment that uses get_https and friends (such as net-ssleay-thread-crash suite posted here recently), you need to initialise the SSL library before threading starts: # NO: #Net::SSLeay::load_error_strings(); #Net::SSLeay::SSLeay_add_ssl_algorithms(); #Net::SSLeay::randomize(); Net::SSLeay::initialize(); # Do this instead Please let me know how you go. Cheers. On Thursday, January 19, 2012 08:51:52 AM you wrote: Show quoted text

> Hi, > > On Wednesday, January 18, 2012 01:46:26 PM you wrote:

> > Queue: Net-SSLeay > > > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > >

> > > Net::SSLeay::get_https reinitialises the library by calling > > > SSLeay_add_ssl_algorithms (which is really SSL_library_init), > > > however, > > > SSL_library_init is not reentrant, according to > > > http://www.openssl.org/docs/ssl/SSL_library_init.html > > > and its seems that overlapping calls cause the double free. > > > > > > If I arrange for the library to be initiliased exactly once, your > > > crash-test runs forever (or at least until the server side runs out > > > of > > > memory: it leaks badly)

> > > > Do you have a guess whether it is my code or Net::SSLeay leaking? The > > thing is that I am more interested in multi-threaded use of Net::SSLeay > > on server side.

> > I have not looked. But I can tell you that I use net-ssleay a lot as a > server without leaks. However I dont use the Net::SSLeay::set_fd interface.

> > > Seems that get_https was written long ago before threading was an > > > issue, and making substantial changes to it now would break > > > existing code. > > > > > > Its not clear to me what is the best way to fix this, though: each > > > thread cant see the perl variables in the other threads, so we cant > > > use > > > a perl flag to indicate that a thread has already initilised the SSL > > > library.

> > > > To share a variable between threads perhaps SvSHARE might come handy > > (see http://perldoc.perl.org/perlapi.html#Magical-Functions) but it will > > probably also require some kind of locking. > >

> > > Seems that the SSL library would have to be initialised once in the > > > _calling_ code before threading starts, but this is going to need a > > > change to calling code, as well as get_https. I cant see a way to > > > contain the fix within SSLeay.pm :-(

> > > > What about to use BOOT: section of SSLeay.xs like this: > > > > 1/ put into BOOT: > > load_error_strings(); > > SSLeay_add_ssl_algorithms(); > > (which - I hope - means that this initialization will done exactly once > > when you "use Net::SSLeay") > > > > 2/ remove all usage of SSLeay_add_ssl_algorithms and load_error_strings > > from *.pm > > > > 3/ in SSLeay.xs replace SSL_library_init and load_error_strings with > > empty subs or deprecation warning

> > Except that some people need to initialise in a different way, and at > different times. > > > My preferred option now is to change SSLeay.pm so: > > 1. The initilaisation code that the SSLeay.pm functions use is moved to a > sub, and the sub uses a flag to only initialise once. This will allow > existing non- threaded code that uses https_cat and sslcat to continue to > run unchanged > > 2. Require code that uses https_cat and sslcat in multiple threads to call > that initialisation function before threading starts. > > Code that does not use https_cat and sslcat would continue to initialise the > library in their current way unchanged, but they could use the new init > function if they wanted. > > I think this means that existing code will continue to run, and the only > code that would have to change is threaded code that uses https_cat and > sslcat (like yours) which does not work correctly anyway. > > Views? > > Cheers. >

> > -- > > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Sun Jan 22 15:00:17 2012 kmx [...] volny.cz - Correspondence added

CC:	info [...] maximka.de
Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Sun, 22 Jan 2012 21:00:03 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

The disadvantage of your approach is that it will not handle a situation when you wanna use Net::SSLeay together with some other Net::SSLeay based perl module like IO::Socket::SSL which is very likely to call Net::SSLeay::SSLeay_add_ssl_algorithms() at some point and therefore will make troubles in threaded app. I would prefer to make SSLeay_add_ssl_algorithms() thread-safe on its own without an extra wrapper function (like in my patch - net-ssleay-thead-safety-proposal1.diff - suggested a couple of post above) - by this we can make "to-some-extent-thread-safe" also other modules based on Net::SSLeay - without the need to switch to the newly created initialize() function. -- kmx

Sun Jan 22 18:03:19 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Mon, 23 Jan 2012 09:03:01 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hello, On Sunday, January 22, 2012 03:00:18 PM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > The disadvantage of your approach is that it will not handle a situation > when you wanna use Net::SSLeay together with some other Net::SSLeay > based perl module like IO::Socket::SSL which is very likely to call > Net::SSLeay::SSLeay_add_ssl_algorithms() at some point and therefore > will make troubles in threaded app.

I dont think thats right. IO::Socket::SSL 1.53 and earlier versions calls the Net::SSLeay initialization once in a BEGIN block, and not repeatedly, so the non-reentrancy of SSLeay_add_ssl_algorithms would not affect it. Provides code that uses both libraries in a threaded environment calls Net::SSLeay::initialize() before threading starts, I would not expect to see a problem. And in a non-threaded environment, even without calling Net::SSLeay::initialize() explicitly, I would expect it to work. Please correct me if I am wrong. AFAICT, the low level Net::SSLeay API is already thread safe. Show quoted text

> > I would prefer to make SSLeay_add_ssl_algorithms() thread-safe on its > own without an extra wrapper function (like in my patch - > net-ssleay-thead-safety-proposal1.diff - suggested a couple of post > above) - by this we can make "to-some-extent-thread-safe" also other > modules based on Net::SSLeay - without the need to switch to the newly > created initialize() function. > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Tue Jan 24 10:41:08 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Tue, 24 Jan 2012 16:40:54 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

>> The disadvantage of your approach is that it will not handle a situation >> when you wanna use Net::SSLeay together with some other Net::SSLeay >> based perl module like IO::Socket::SSL which is very likely to call >> Net::SSLeay::SSLeay_add_ssl_algorithms() at some point and therefore >> will make troubles in threaded app. >>

> I dont think thats right. > IO::Socket::SSL 1.53 and earlier versions calls the Net::SSLeay initialization > once in a BEGIN block, and not repeatedly, so the non-reentrancy of > SSLeay_add_ssl_algorithms would not affect it. > > Provides code that uses both libraries in a threaded environment calls > Net::SSLeay::initialize() before threading starts, I would not expect to see a > problem. And in a non-threaded environment, even without calling > Net::SSLeay::initialize() explicitly, I would expect it to work. > > Please correct me if I am wrong. >

Yes, it seems that you are right as for the IO::Socket::SSL, but keep in mind that there are many others using Net::SSLeay, for example http://grep.cpan.me/?q=Net%3A%3ASSLeay%3A%3ASSLeay_add_ssl_algorithms Let us have a scenario: [myapp.pl] use IO::Socket::SSL; #based on Net::SSLeay use Something::Else::SSL; #also based on Net::SSLeay ... [end] if both IO::Socket::SSL and Something::Else::SSL do in their BEGIN block: Net::SSLeay::load_error_strings(); Net::SSLeay::SSLeay_add_ssl_algorithms(); Net::SSLeay::randomize(); or if there is Something::Else::SSL->get_https calling Net::SSLeay::SSLeay_add_ssl_algorithms() Then we have double call of Net::SSLeay::SSLeay_add_ssl_algorithms which should be avoided. Show quoted text

> AFAICT, the low level Net::SSLeay API is already thread safe. >

1/ I am not sure about re-entrance of OPENSSL_add_all_algorithms_noconf() used inside CTX_use_PKCS12_file() 2/ We are still not following http://www.openssl.org/docs/crypto/threads.html "OpenSSL can safely be used in multi-threaded applications provided that at least two callback functions are set, locking_function and threadid_func. " -- kmx

Tue Jan 24 20:55:10 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 25 Jan 2012 11:54:46 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

On Tuesday, January 24, 2012 10:41:10 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >

> >> The disadvantage of your approach is that it will not handle a > >> situation > >> when you wanna use Net::SSLeay together with some other Net::SSLeay > >> based perl module like IO::Socket::SSL which is very likely to call > >> Net::SSLeay::SSLeay_add_ssl_algorithms() at some point and therefore > >> will make troubles in threaded app.

> > > > I dont think thats right. > > IO::Socket::SSL 1.53 and earlier versions calls the Net::SSLeay > > initialization once in a BEGIN block, and not repeatedly, so the > > non-reentrancy of SSLeay_add_ssl_algorithms would not affect it. > > > > Provides code that uses both libraries in a threaded environment calls > > Net::SSLeay::initialize() before threading starts, I would not expect to > > see a problem. And in a non-threaded environment, even without calling > > Net::SSLeay::initialize() explicitly, I would expect it to work. > > > > Please correct me if I am wrong.

> > Yes, it seems that you are right as for the IO::Socket::SSL, but keep in > mind that there are many others using Net::SSLeay, for example > http://grep.cpan.me/?q=Net%3A%3ASSLeay%3A%3ASSLeay_add_ssl_algorithms

That is precisely why I am reluctant to make changes to the way it currently does its default initialisation: for fear of breaking other existing libraries and implementations. I have tried to be very conservative with the change I have made, with the expectation that currently working applications will not be affected, and that only currently non-working or buggy apps will have to change. Show quoted text

> > Let us have a scenario: > > [myapp.pl] > use IO::Socket::SSL; #based on Net::SSLeay > use Something::Else::SSL; #also based on Net::SSLeay > ... > [end] > > if both IO::Socket::SSL and Something::Else::SSL do in their BEGIN block: > Net::SSLeay::load_error_strings(); > Net::SSLeay::SSLeay_add_ssl_algorithms(); > Net::SSLeay::randomize(); > or if there is Something::Else::SSL->get_https calling > Net::SSLeay::SSLeay_add_ssl_algorithms() > > Then we have double call of Net::SSLeay::SSLeay_add_ssl_algorithms which > should be avoided.

Perhaps, but if IO::Socket::SSL or some other library initialises openssl while Net::SSLeay is not looking, there not much to be done. Net::SSLeay is unable to tell if some other library has already (or will sometime later) initialise the openssl library. Show quoted text

>

> > AFAICT, the low level Net::SSLeay API is already thread safe.

> > 1/ I am not sure about re-entrance of > OPENSSL_add_all_algorithms_noconf() used inside CTX_use_PKCS12_file() > > 2/ We are still not following > http://www.openssl.org/docs/crypto/threads.html > "OpenSSL can safely be used in multi-threaded applications provided that > at least two callback functions are set, locking_function and > threadid_func. "

AFAICS, there is no locking used by the relevant functions within SSLeay_add_ssl_algorithms so adding this type of locking support would have no impact on the problem originally reported, which would still have to be solved (and , AFAICS, has been solved at least for Net::SSLeay using the preferred method previously discussed) Crypt::SSLeay, like Net::SSLeay does not set up any locking function either. Im thinking that it is not necessary to provide these lock functions when openssl is embedded in perl, because perl's so-called threading is cooperative rather than preemptive, and therefore threads can only switch when perl is in control (and not inside the ssl functions, where locking is used). Show quoted text

> > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Wed Jan 25 16:36:29 2012 kmx [...] volny.cz - Correspondence added

CC:	info [...] maximka.de
Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 25 Jan 2012 22:36:16 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> Perhaps, but if IO::Socket::SSL or some other library initialises openssl > while Net::SSLeay is not looking, there not much to be done. Net::SSLeay is > unable to tell if some other library has already (or will sometime later) > initialise the openssl library. >

I think Net::SSLeay is able to allow exactly one call of SSL_library_init per process (even multithreaded) if we do (in SSLeay.xs) something like this: int SSL_library_init() ALIAS: SSLeay_add_ssl_algorithms = 1 OpenSSL_add_ssl_algorithms = 2 add_ssl_algorithms = 3 CODE: dMY_CXT; #ifdef USE_ITHREADS MUTEX_LOCK(&MY_CXT.init_mutex); #endif if (!MY_CXT.initialized) { SSL_library_init(); MY_CXT.initialized = 1; } #ifdef USE_ITHREADS MUTEX_UNLOCK(&MY_CXT.init_mutex); #endif Then we will handle 100% of situations when any other Net::SSLeay based modules repeatedly call Net::SSLeay::SSLeay_add_ssl_algorithms() Show quoted text

>>> AFAICT, the low level Net::SSLeay API is already thread safe. >>>

>> 1/ I am not sure about re-entrance of >> OPENSSL_add_all_algorithms_noconf() used inside CTX_use_PKCS12_file() >> >> 2/ We are still not following >> http://www.openssl.org/docs/crypto/threads.html >> "OpenSSL can safely be used in multi-threaded applications provided that >> at least two callback functions are set, locking_function and >> threadid_func. " >>

> AFAICS, there is no locking used by the relevant functions within > SSLeay_add_ssl_algorithms so adding this type of locking support would have no > impact on the problem originally reported, which would still have to be solved > (and , AFAICS, has been solved at least for Net::SSLeay using the preferred > method previously discussed) > > Crypt::SSLeay, like Net::SSLeay does not set up any locking function either. >

Yes but there is at least a discussion to this topic in Crypt::SSLeay's RT https://rt.cpan.org/Public/Bug/Display.html?id=41007 Show quoted text

> Im thinking that it is not necessary to provide these lock functions when > openssl is embedded in perl, because perl's so-called threading is cooperative > rather than preemptive, and therefore threads can only switch when perl is in > control (and not inside the ssl functions, where locking is used). >

Maybe, I am not an expert on threads-related perl guts. I am slightly in doubts whether your assumptions are also valid for multi core CPUs. To be honest I am not able to demonstrate a crash due to missing locking_function. I am just saying that we are not following openssl's thread-safety best practice. I agree to leave "missing-locking_function-issue" until somebody is able to create a proof of concept crash test. However, another source of thread-unsafety troubles might be static variables in SSLeay.xs used for various callbacks. See chapter recommended best practice (which we do not follow) at http://perldoc.perl.org/perlxs.html#Safely-Storing-Static-Data-in-XS -- kmx

Wed Jan 25 17:40:07 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 26 Jan 2012 08:39:46 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, On Wednesday, January 25, 2012 04:36:30 PM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >

> > Perhaps, but if IO::Socket::SSL or some other library initialises > > openssl > > while Net::SSLeay is not looking, there not much to be done. Net::SSLeay > > is unable to tell if some other library has already (or will sometime > > later) initialise the openssl library.

> > I think Net::SSLeay is able to allow exactly one call of > SSL_library_init per process (even multithreaded) if we do (in > SSLeay.xs) something like this:

interesting idea. Have you tested it? Can you send a unified diff? Show quoted text

> > int > SSL_library_init() > ALIAS: > SSLeay_add_ssl_algorithms = 1 > OpenSSL_add_ssl_algorithms = 2 > add_ssl_algorithms = 3 > CODE: > dMY_CXT; > #ifdef USE_ITHREADS > MUTEX_LOCK(&MY_CXT.init_mutex); > #endif > if (!MY_CXT.initialized) { > SSL_library_init(); > MY_CXT.initialized = 1; > } > #ifdef USE_ITHREADS > MUTEX_UNLOCK(&MY_CXT.init_mutex); > #endif > > > Then we will handle 100% of situations when any other Net::SSLeay based > modules repeatedly call Net::SSLeay::SSLeay_add_ssl_algorithms() >

> >>> AFAICT, the low level Net::SSLeay API is already thread safe.

> >> > >> 1/ I am not sure about re-entrance of > >> OPENSSL_add_all_algorithms_noconf() used inside CTX_use_PKCS12_file() > >> > >> 2/ We are still not following > >> http://www.openssl.org/docs/crypto/threads.html > >> "OpenSSL can safely be used in multi-threaded applications provided > >> that > >> at least two callback functions are set, locking_function and > >> threadid_func. "

> > > > AFAICS, there is no locking used by the relevant functions within > > SSLeay_add_ssl_algorithms so adding this type of locking support would > > have no impact on the problem originally reported, which would still > > have to be solved (and , AFAICS, has been solved at least for > > Net::SSLeay using the preferred method previously discussed) > > > > Crypt::SSLeay, like Net::SSLeay does not set up any locking function > > either.

> Yes but there is at least a discussion to this topic in Crypt::SSLeay's > RT https://rt.cpan.org/Public/Bug/Display.html?id=41007 >

> > Im thinking that it is not necessary to provide these lock functions > > when > > openssl is embedded in perl, because perl's so-called threading is > > cooperative rather than preemptive, and therefore threads can only > > switch when perl is in control (and not inside the ssl functions, where > > locking is used).

> Maybe, I am not an expert on threads-related perl guts. I am slightly in > doubts whether your assumptions are also valid for multi core CPUs. > > To be honest I am not able to demonstrate a crash due to missing > locking_function. I am just saying that we are not following openssl's > thread-safety best practice. I agree to leave > "missing-locking_function-issue" until somebody is able to create a > proof of concept crash test. > > However, another source of thread-unsafety troubles might be static > variables in SSLeay.xs used for various callbacks. See chapter > recommended best practice (which we do not follow) at > http://perldoc.perl.org/perlxs.html#Safely-Storing-Static-Data-in-XS > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Thu Jan 26 02:25:53 2012 kmx [...] volny.cz - Correspondence added

CC:	info [...] maximka.de
Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 26 Jan 2012 08:25:39 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> interesting idea. Have you tested it? Can you send a unified diff? >

See attached net-ssleay-thead-safety-proposal2.diff I have tested it but only on perl 5.14-with-ithreads (but it should work also on older perls and perl without ithreads). -- kmx

Message body is not shown because sender requested not to inline it.

Thu Jan 26 14:14:57 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 26 Jan 2012 20:14:46 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Sorry, my previous patch was not working as I have expected, please check enclosed net-ssleay-thead-safety-proposal3.diff -- kmx

Message body is not shown because sender requested not to inline it.

Thu Jan 26 17:39:06 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Fri, 27 Jan 2012 08:38:44 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hello, Thanks for t hat. I have tested your mutex patch on several platforms here including windows, and also against the net-ssleay-thread-crash suite, with and without calling Net::SSLeay::initialize in main. Works with all my tests so far, so I have committed your patch to SVN. Please test in as many environments as you can and let me know how you go. Cheers. On Thursday, January 26, 2012 02:14:59 PM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > Sorry, my previous patch was not working as I have expected, please > check enclosed net-ssleay-thead-safety-proposal3.diff > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Fri Jan 27 08:13:46 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Fri, 27 Jan 2012 14:13:35 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> Please test in as many environments as you can and let me know how you go. >

Will do. Meantime please consider attached documentation patch. -- kmx

Message body is not shown because sender requested not to inline it.

Fri Jan 27 08:38:06 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Fri, 27 Jan 2012 14:37:55 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Bad news: another multi-threading related crash test - this one is related to callbacks. Check enclosed net-ssleay-callback-crash.tar.gz Crashes nearly immediately (tested with current svn trunk version). -- kmx

Download net-ssleay-callback-crash.tar.gz
application/x-gzip 5.9k

Message body not shown because it is not plain text.

Fri Jan 27 09:22:54 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Fri, 27 Jan 2012 15:22:42 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Good news: I have the first proposal for fixing crashing callbacks in multi-threaded Net::SSLeay application. Check callback-threading-patch1.diff It seems to work, I am only not sure whether CLONE() implementation does what we really need (to be honest it my first CLONE() routine implemented in XS - feel free to correct me). It definitely deserves more testing. -- kmx

Message body is not shown because sender requested not to inline it.

Fri Jan 27 10:00:14 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Fri, 27 Jan 2012 16:00:03 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

The last multi-threading related test suite for this week regards CTX_new() In net-ssleay-ctx_new-timing.tar.gz you will find 2 nearly the same scripts ctx-test1.pl ctx-test2.pl On my Windows (32bit/perl 5.14.2) box: - ctx-test1.pl runs approx 3 seconds - ctx-test2.pl runs couple of minutes (I have not tested on other platforms) Both scripts start 50 threads doing SSLeay::CTX_new() + Net::SSLeay::CTX_free() the only difference is that ctx-test1.pl calls also once SSLeay::CTX_new() in the main thread before threading actually starts. The workaround is "call at least once SSLeay::CTX_new() in the main thread before creating any child thread" (which is fine for me) - but it is strange. Any idea? -- kmx

Download net-ssleay-ctx_new-timing.tar.gz
application/x-gzip 653b

Message body not shown because it is not plain text.

Fri Jan 27 13:50:50 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Fri, 27 Jan 2012 19:50:38 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

On 27.1.2012 16:00, kmx via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=73532> > > The last multi-threading related test suite for this week regards CTX_new() > > In net-ssleay-ctx_new-timing.tar.gz you will find 2 nearly the same > scripts ctx-test1.pl ctx-test2.pl > > On my Windows (32bit/perl 5.14.2) box: > - ctx-test1.pl runs approx 3 seconds > - ctx-test2.pl runs couple of minutes > (I have not tested on other platforms) > > Both scripts start 50 threads doing SSLeay::CTX_new() + > Net::SSLeay::CTX_free() the only difference is that ctx-test1.pl calls > also once SSLeay::CTX_new() in the main thread before threading actually > starts. >

Further investigation revealed that cxt-test2.pl gets stuck inside openssl's function SSL_CTX *SSL_CTX_new(const SSL_METHOD *method) - perhaps a kind of a deadlock inside openssl library. And by quick and dirty implementation (I have to polish it a bit) of locking described at: http://www.openssl.org/docs/crypto/threads.html I was able to fix the deadlock in ctx-test2.pl So this is IMHO a reason why we need to implement it -- kmx

Fri Jan 27 18:23:39 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Sat, 28 Jan 2012 09:23:20 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, On Friday, January 27, 2012 10:00:15 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > The last multi-threading related test suite for this week regards CTX_new() > > In net-ssleay-ctx_new-timing.tar.gz you will find 2 nearly the same > scripts ctx-test1.pl ctx-test2.pl > > On my Windows (32bit/perl 5.14.2) box: > - ctx-test1.pl runs approx 3 seconds > - ctx-test2.pl runs couple of minutes > (I have not tested on other platforms)

Strange, Quick test here on Linux shows both run to completion in about 3 secs What is it doing during those 2 minutes? Show quoted text

> > Both scripts start 50 threads doing SSLeay::CTX_new() + > Net::SSLeay::CTX_free() the only difference is that ctx-test1.pl calls > also once SSLeay::CTX_new() in the main thread before threading actually > starts. > > The workaround is "call at least once SSLeay::CTX_new() in the main > thread before creating any child thread" (which is fine for me) - but it > is strange. > > Any idea? > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Fri Jan 27 18:40:07 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Sat, 28 Jan 2012 09:39:49 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, On Friday, January 27, 2012 08:13:48 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >

> > Please test in as many environments as you can and let me know how you > > go.

> Will do. > > Meantime please consider attached documentation patch.

Adopted. Thanks. Show quoted text

> > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Fri Jan 27 20:18:43 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Sat, 28 Jan 2012 11:18:24 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, your patch works for me on a range of platforms and perl versions. Adopted and posted to svn. On Friday, January 27, 2012 09:22:55 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > Good news: I have the first proposal for fixing crashing callbacks in > multi-threaded Net::SSLeay application. > > Check callback-threading-patch1.diff > > It seems to work, I am only not sure whether CLONE() implementation does > what we really need (to be honest it my first CLONE() routine > implemented in XS - feel free to correct me). > > It definitely deserves more testing. > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Sat Jan 28 05:58:27 2012 kmx [...] volny.cz - Correspondence added

CC:	info [...] maximka.de
Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Sat, 28 Jan 2012 11:58:17 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> Strange, Quick test here on Linux shows both run to completion in about 3 secs > > What is it doing during those 2 minutes? >

50 threads running, consuming 50% of my dual-core CPU (~ 1 core), the first threads finishes after approx. 50sec, the second after another approx. 20sec, ... (it is not 2 minutes, I meant "minutes" like many minutest - in fact I have never wait for all 50 threads to finish as I expect it to run for at least half an hour) My guess is - 1 thread looping in some kind of nearly-deadlock-situation (occupiing 1 core) - 49 threads waiting (for something) -- kmx

Sun Jan 29 16:06:22 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Sun, 29 Jan 2012 22:06:08 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Please check enclosed tarball with 2 tests: - 61_threads-cb-crash.t (passes with the latest SVN trunk version) - 62_threads-ctx_new-deadlock.t (fails on my Win32 box - as it does not finish within 20s timeout) Consider adding these to Net-SSLeay distribution. -- kmx

Download thread-tests1.tar.gz
application/x-gzip 1k

Message body not shown because it is not plain text.

Mon Jan 30 04:33:28 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Mon, 30 Jan 2012 10:33:15 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

I have prepared a patch fixing CTX_new() deadlock I have experienced on my MS Windows machine. - I have tried to implement all locking parts required and described at http://www.openssl.org/docs/crypto/threads.html - I did my best to keep the implementation as portable as possible (I am using perl_mutex structure which should be portable by its own) - I have used couple of "#if OPENSSL_VERSION_NUMBER ..." to assure that we are turning on openssl locking only on versions that support it - However I have tested (successfully) the implementation only on MS Windows 32bit and 64bit More testing needed - especially on Linux (and/or other UNIXes) box + ithreads enabled + multicore CPU (I have mostly access to 1core/cpu virtual machines) - it would be nice to test on a box with openssl-1.0.0+ and pre-1.0.0 (as some parts of implementation differs for 0.9.x / 1.x.x) - peer review of openssl_threadid_func() implementation is welcome (this is a point where I am not sure - marked by "XXX-FIXME note by KMX" in source code) - please note that openssl_threadid_func() is not used at all on MS Windows thus I am really not sure whether it will work as expected If you are against applying this kind of patch for all platforms I would like to ask you to make it available at least for Win32 (by #ifdef WIN32) as it is really necessary to avoid CTX_new() deadlock described in my previous post. -- kmx

Message body is not shown because sender requested not to inline it.

Mon Jan 30 06:58:17 2012 info [...] maximka.de - Correspondence added

Hi, Show quoted text

> More testing needed

here my test result for a Linux system: debian squeeze amd64 4 cpu-cores openssl 0.9.8o-4squeeze7 perl v5.10.1 perl t/local/61_threads-cb-crash.t 1..1 Unbalanced string table refcount: (1) for "32936256" during global destruction.Unbalanced string table refcount: (1) for "35428992" during global destruction.Unbalanced string table refcount: (1) for "39702592" during global destruction. Unbalanced string table refcount: (1) for "33308880" during global destruction. Unbalanced string table refcount: (1) for "35549232" during global destruction. Unbalanced string table refcount: (1) for "32933408" during global destruction. Unbalanced string table refcount: (1) for "33274160" during global destruction. Segmentation fault t/local/62_threads-ctx_new-deadlock.t .. ok regards, palik

Mon Jan 30 07:32:04 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Mon, 30 Jan 2012 13:31:54 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

palik, have you used the latest SVN trunk version (rev287) ? It should already contain the fix for crashing 61_threads-cb-crash.t If you have used the latest version we probably have another multithreaded issue to fix. -- kmx

Mon Jan 30 08:05:50 2012 info [...] maximka.de - Correspondence added

Show quoted text

> have you used the latest SVN trunk version (rev287) ?

No, I used from CPAN downloaded tarball, because I couldn't found SVN URL :-( palik

Mon Jan 30 08:31:05 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Mon, 30 Jan 2012 14:30:51 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> No, I used from CPAN downloaded tarball, because I couldn't found SVN URL :-( >

svn://svn.debian.org/net-ssleay/trunk

Mon Jan 30 08:53:38 2012 info [...] maximka.de - Correspondence added

On Mon Jan 30 08:31:05 2012, kmx@volny.cz wrote: Show quoted text

>

> > No, I used from CPAN downloaded tarball, because I couldn't found

> SVN URL :-(

> >

> > svn://svn.debian.org/net-ssleay/trunk

Hm, can't connect. I'll try again in the evening at home. host svn.debian.org svn.debian.org has address 217.196.43.140 svn co svn://svn.debian.org/net-ssleay/trunk net-ssleay-trunk svn: Can't connect to host 'svn.debian.org': Operation timed out palik

Mon Jan 30 13:14:48 2012 info [...] maximka.de - Correspondence added

On Mon Jan 30 08:31:05 2012, kmx@volny.cz wrote: Show quoted text

>

> > No, I used from CPAN downloaded tarball, because I couldn't found

> SVN URL :-(

> >

> > svn://svn.debian.org/net-ssleay/trunk >

still can't connect: svn: Can't connect to host 'svn.debian.org': Network is unreachable palik

Mon Jan 30 14:01:17 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Mon, 30 Jan 2012 20:01:07 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> still can't connect: > svn: Can't connect to host 'svn.debian.org': Network is unreachable >

The same here :( -- kmx

Mon Jan 30 14:06:49 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Mon, 30 Jan 2012 20:06:40 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> If you are against applying this kind of patch for all platforms I would > like to ask you to make it available at least for Win32 (by #ifdef > WIN32) as it is really necessary to avoid CTX_new() deadlock described > in my previous post. >

Or another fallback solution you might like more could be introducing a new function Net::SSLeay::init_openssl_tread_safe_locking() which will enable the locking-kung-fu I have sent a couple of posts before. It will probably need one more global mutex to avoid multiple initialization but it is something I know how to handle. -- kmx

Mon Jan 30 16:27:26 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Tue, 31 Jan 2012 07:27:11 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hello, On Sunday, January 29, 2012 04:06:24 PM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > Please check enclosed tarball with 2 tests: > - 61_threads-cb-crash.t (passes with the latest SVN trunk version) > - 62_threads-ctx_new-deadlock.t (fails on my Win32 box - as it does not > finish within 20s timeout) > > Consider adding these to Net-SSLeay distribution.

I would like to do that except your tests do not compile on perl 5.8 and others because threads::running is not available as an argument to list() Cheers. Show quoted text

> > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Mon Jan 30 16:32:13 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Tue, 31 Jan 2012 07:31:59 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hello, I see similar problems ONLY if I run the tests against the non-fixed net- ssleay On Monday, January 30, 2012 06:58:18 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > Hi, >

> > More testing needed

> > here my test result for a Linux system: > debian squeeze amd64 4 cpu-cores > openssl 0.9.8o-4squeeze7 > perl v5.10.1 > > perl t/local/61_threads-cb-crash.t

Are you sure you are using the newly compiled net-ssleay? Maybe you need: perl -w -I blib/lib -I blib/arch t/local/61_threads-cb-crash. Cheers. Show quoted text

> 1..1 > Unbalanced string table refcount: (1) for "32936256" during global > destruction.Unbalanced string table refcount: (1) for "35428992" during > global destruction.Unbalanced string table refcount: (1) for "39702592" > during global destruction. > Unbalanced string table refcount: (1) for "33308880" during global > destruction. Unbalanced string table refcount: (1) for "35549232" during > global destruction. Unbalanced string table refcount: (1) for "32933408" > during global destruction. Unbalanced string table refcount: (1) for > "33274160" during global destruction. Segmentation fault > > t/local/62_threads-ctx_new-deadlock.t .. ok > > > regards, > palik

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Mon Jan 30 16:49:31 2012 info [...] maximka.de - Correspondence added

Hello, after checkout svn code (Revision: 287) both new tests are ok. t/local/61_threads-cb-crash.t .......... ok t/local/62_threads-ctx_new-deadlock.t .. ok But I couldn't apply your patch to SSLeay.xs patch --dry-run SSLeay.xs ../net-ssleay/net-ssleay-thead-safety-proposal1.diff patching file SSLeay.xs Hunk #1 succeeded at 92 with fuzz 2 (offset 18 lines). Hunk #2 succeeded at 715 with fuzz 2 (offset 54 lines). Hunk #3 FAILED at 1366. think to add requires('Module::Install') into Makefile.PL regards, palik

Mon Jan 30 19:19:43 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Tue, 31 Jan 2012 10:19:26 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi. A mixed bag of results here using your threads patch and the cb-crash.pl you earlier posted (but using a different certificate that 0.9.8i could understand): Linux perl 5.14, openssl 1.0.0. Perfect. (but, I must say, it ran perfectly even before your locking patch) Windows perl 5.12, openssl 0.9.8i, mostly OK, but occasional error: Thread 925 terminated abnormally: CTX_use_PrivateKey_file failed at cb_crash.pl line 50 Windows perl 5.10, openssl 0.9.8i, mostly OK, but occasional error: Attempt to free non-existent shared string 'KILL', Perl interpreter: 0xd24268c during global destruction. Windows perl 5.8.8, openssl 0.9.8i, ran fine until thread 0602, when it started to block and hang. 'Popup Application Error' messages with errors showing perl trying to write to 0x00000000. Windows perl 5.6, openssl 0.9.8i, Could not test because threads.pm not available So at this stage if have backed you patch out from my working copy and will not commit it yet. Cheers. On Monday, January 30, 2012 04:33:29 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > I have prepared a patch fixing CTX_new() deadlock I have experienced on > my MS Windows machine. > - I have tried to implement all locking parts required and described at > http://www.openssl.org/docs/crypto/threads.html > - I did my best to keep the implementation as portable as possible (I am > using perl_mutex structure which should be portable by its own) > - I have used couple of "#if OPENSSL_VERSION_NUMBER ..." to assure that > we are turning on openssl locking only on versions that support it > - However I have tested (successfully) the implementation only on MS > Windows 32bit and 64bit > > More testing needed > - especially on Linux (and/or other UNIXes) box + ithreads enabled + > multicore CPU (I have mostly access to 1core/cpu virtual machines) > - it would be nice to test on a box with openssl-1.0.0+ and pre-1.0.0 > (as some parts of implementation differs for 0.9.x / 1.x.x) > - peer review of openssl_threadid_func() implementation is welcome (this > is a point where I am not sure - marked by "XXX-FIXME note by KMX" in > source code) > - please note that openssl_threadid_func() is not used at all on MS > Windows thus I am really not sure whether it will work as expected > > If you are against applying this kind of patch for all platforms I would > like to ask you to make it available at least for Win32 (by #ifdef > WIN32) as it is really necessary to avoid CTX_new() deadlock described > in my previous post. > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Tue Jan 31 02:59:10 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Tue, 31 Jan 2012 08:58:58 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> But I couldn't apply your patch to SSLeay.xs >

Try enclosed patch fix-for-ctx_new-deadlock-v2_rev288.diff which should work with current trunk (revision 288). -- kmx

Message body is not shown because sender requested not to inline it.

Tue Jan 31 03:09:49 2012 info [...] maximka.de - Correspondence added

Show quoted text

> Try enclosed patch fix-for-ctx_new-deadlock-v2_rev288.diff which should > work with current trunk (revision 288).

It's ok now. t/local/61_threads-cb-crash.t .......... ok t/local/62_threads-ctx_new-deadlock.t .. ok palik

Tue Jan 31 03:21:34 2012 kmx [...] volny.cz - Correspondence added

CC:	info [...] maximka.de
Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Tue, 31 Jan 2012 09:21:23 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> Windows perl 5.12, openssl 0.9.8i, mostly OK, but occasional error ... > Windows perl 5.10, openssl 0.9.8i, mostly OK, but occasional error ... > Windows perl 5.8.8, openssl 0.9.8i, ran fine until thread 0602 ... >

Well, not good; however [irony]still better then unpatched v1.42[/irony] Could you please check whether it makes a difference to run crash-test 1/ with current trunk vs. 2/ trunk+fix-for-ctx_new-deadlock-v2_rev288.diff ? In other words does the newly proposed openssl-locking-kung-fu make it worse ? My environment is: - perl 5.14.2 - OpenSSL 1.0.0d - threads-1.86 - gcc compiler 4.4.7-pre + c-runtime from mingw-w64.sf.net project - all of that for 32bit as well as 64bit MS Windows - on both 32/64bit cb-crash.pl repeatedly runs without any crash I will try to prepare a crashy environment - but it will take some time (unfortunately gcc+mingw runtime from mingw-w64.sf.net I am using does not build 5.10.x and earlier without patching perl source code so I should perhaps take some MS compiler + rebuild some of the older perl as well as some of the older openssl lib). Or maybe could you give more info about your perl 5.8.8 (if you consider this to be the oldest supported perl by Net::SSLeay) - compiler version, threads version, vendor (activestate or homebrewed?) -- kmx

Tue Jan 31 10:55:55 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Tue, 31 Jan 2012 16:55:39 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

mike and palik, could you please try to run the new attached callback-crash-suite on your perl boxes? it contains 8 scripts: - cb-crash-ng0-nossleay.pl - cb-crash-ng1.pl - cb-crash-ng2.pl - cb-crash-ng2-nopasswd.pl - cb-crash-ng3.pl - cb-crash-ng3-nopasswd.pl - cb-crash-ng4.pl - cb-crash-ng4-nopassw.pl it would be nice to have for all 8 scripts: 1/ results for current trunk (rev288) 2/ results for trunk+fix-for-ctx_new-deadlock-v2_rev288.diff On my MSWin 32bit just cb-crash-ng4.pl (cross-thread callback) crashes all the other run and finish fine. Thanks in advance for your time. -- kmx

Download net-ssleay-callback-crash2.tar.gz
application/x-gzip 2.7k

Message body not shown because it is not plain text.

Tue Jan 31 12:43:03 2012 info [...] maximka.de - Correspondence added

Show quoted text

> it would be nice to have for all 8 scripts: > 1/ results for current trunk (rev288) > 2/ results for trunk+fix-for-ctx_new-deadlock-v2_rev288.diff

execution outputs are in attachment. Show quoted text

> > On my MSWin 32bit just cb-crash-ng4.pl (cross-thread callback) crashes > all the other run and finish fine. >

in my case cb-crash-ng2.pl and cb-crash-ng4.pl died at CTX_use_PrivateKey_file call line 17 palik

Subject:

net-ssleay-callback-crash2-result.tar.gz

Download net-ssleay-callback-crash2-result.tar.gz
application/x-gzip 675.7k

Message body not shown because it is not plain text.

Tue Jan 31 14:33:11 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Tue, 31 Jan 2012 20:32:55 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

thanks, it is still: debian squeeze amd64 4 cpu-cores openssl 0.9.8o-4squeeze7 perl v5.10.1 right? conclusion from your testing: 1/ my encrypted test key pk_1024.key.pem probably does not work with your openssl 2/ the new openssl-locking-kung-fu patch does not make things worse on Linux ad 1/ could you please check the encrypted private key (password is "123") [testbox]$ openssl rsa -in pk_1024.key.pem -text perhaps try to change password to something longer then 3 chars by: [testbox]$ openssl rsa -in pk_1024.key.pem -out pk_1024-new.key.pem -des (+ patch cb-crash-ng2.pl cb-crash-ng3.pl cb-crash-ng4.pl - pk_1024-new.key.pem + new password) -- kmx

Tue Jan 31 20:39:06 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 11:38:51 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, On Tuesday, January 31, 2012 03:21:35 AM kmx via RT wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >

> > Windows perl 5.12, openssl 0.9.8i, mostly OK, but occasional error ... > > Windows perl 5.10, openssl 0.9.8i, mostly OK, but occasional error ... > > Windows perl 5.8.8, openssl 0.9.8i, ran fine until thread 0602 ...

> > Well, not good; however [irony]still better then unpatched v1.42[/irony] > > Could you please check whether it makes a difference to run crash-test > 1/ with current trunk vs. 2/ trunk+fix-for-ctx_new-deadlock-v2_rev288.diff ?

------------------With current trunk 288 (btw, did you notice that, contrary to my earlier statement, 288 has your earlier thread locking code) Linux perl 5.14, openssl 1.0.0. gcc 4.6.2 on OpenSuSE 12.1 8 core Perfect. Ran to completion. Windows Server 2003 32 bit activeperl 5.12, openssl 0.9.8i, MS Visual C 6, MS C compiler version 12 one error: Thread 45 terminated abnormally: CTX_use_PrivateKey_file failed at cb_crash.pl line 50 and then ran to completion. Windows Server 2003 32 bit activeperl 5.10, openssl 0.9.8i, MS Visual C 6, MS C compiler version 12 one error: Unbalanced string table refcount: (1) for "Net" during global destruction then much later crash after thread 0816 attempt to write to 0x00000000 Windows Server 2003 32 bit activeperl 5.8.8, openssl 0.9.8i, MS Visual C 6, MS C compiler version 12 Had to modify cb_crash test not to use threads::running Crashed after thread 274 occasional error: Thread 925 terminated abnormally: CTX_use_PrivateKey_file failed at cb_crash.pl line 50 and then hung later after: [thread:9996] do_check finished (prob because of ::running change above) Windows Server 2003 32 bit activeperl 5.6, openssl 0.9.8i, MS Visual C 6, MS C compiler version 12 Could not test because threads.pm not available -------------------With your fix-for-ctx_new-deadlock-v2_rev288.diff applied to trunk 288: Linux perl 5.14, openssl 1.0.0. gcc 4.6.2 on OpenSuSE 12.1 8 core Perfect. Ran to completion Windows Server 2003 32 bit activeperl 5.12, openssl 0.9.8i, MS Visual C 6, MS C compiler version 12 error: Thread 833 terminated abnormally: CTX_use_PrivateKey_file failed at cb_crash.pl line 50 then ran to completion Windows Server 2003 32 bit activeperl 5.10, openssl 0.9.8i, MS Visual C 6, MS C compiler version 12 2 error messages: Attempt to free unreferenced glob pointers unbalanced string table refcount: (1) for "c:/perl-5.10/lib/Exporter.pm" during global destruction Crashed later after thread 1488 with write to 0x000000 Windows Server 2003 32 bit activeperl 5.8.8, openssl 0.9.8i, MS Visual C 6, MS C compiler version 12 Had to modify cb_crash test not to use threads::running Crashed after thread 274 Windows Server 2003 32 bit activeperl 5.6, openssl 0.9.8i, MS Visual C 6, MS C compiler version 12 Could not test because threads.pm not available So, I think this patch has not materially improved things on these platforms. I have not adopted it yet. Cheers. Show quoted text

> > In other words does the newly proposed openssl-locking-kung-fu make it > worse ? > > My environment is: > - perl 5.14.2 > - OpenSSL 1.0.0d > - threads-1.86 > - gcc compiler 4.4.7-pre + c-runtime from mingw-w64.sf.net project > - all of that for 32bit as well as 64bit MS Windows > - on both 32/64bit cb-crash.pl repeatedly runs without any crash > > I will try to prepare a crashy environment - but it will take some time > (unfortunately gcc+mingw runtime from mingw-w64.sf.net I am using does > not build 5.10.x and earlier without patching perl source code so I > should perhaps take some MS compiler + rebuild some of the older perl as > well as some of the older openssl lib). > > Or maybe could you give more info about your perl 5.8.8 (if you consider > this to be the oldest supported perl by Net::SSLeay) - compiler version, > threads version, vendor (activestate or homebrewed?) > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Tue Jan 31 21:21:47 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 12:21:31 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, I have a similar problem with cb-crash-ng2.pl and cb-crash-ng4.pl died at CTX_use_PrivateKey_file call line 17 Problem is unknown algorithms. How did you generate the key files? I wish to regenerate them here using algorithms that 0.9.8i understands. Cheers. On Tuesday, January 31, 2012 02:33:13 PM kmx via RT wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > thanks, it is still: > > debian squeeze amd64 4 cpu-cores > openssl 0.9.8o-4squeeze7 > perl v5.10.1 > > right? > > conclusion from your testing: > 1/ my encrypted test key pk_1024.key.pem probably does not work with > your openssl > 2/ the new openssl-locking-kung-fu patch does not make things worse on Linux > > ad 1/ could you please check the encrypted private key (password is "123") > [testbox]$ openssl rsa -in pk_1024.key.pem -text > > perhaps try to change password to something longer then 3 chars by: > [testbox]$ openssl rsa -in pk_1024.key.pem -out pk_1024-new.key.pem -des > (+ patch cb-crash-ng2.pl cb-crash-ng3.pl cb-crash-ng4.pl - > pk_1024-new.key.pem + new password) > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Wed Feb 01 02:07:36 2012 info [...] maximka.de - Correspondence added

Show quoted text

> debian squeeze amd64 4 cpu-cores > openssl 0.9.8o-4squeeze7 > perl v5.10.1 > > right?

yes Show quoted text

> ad 1/ could you please check the encrypted private key (password is "123") > [testbox]$ openssl rsa -in pk_1024.key.pem -text

on squeeze: openssl rsa -in pk_1024.key.pem -text Enter pass phrase for pk_1024.key.pem: 32373:error:28069065:lib(40):UI_set_result:result too small:ui_lib.c:850:You must type in 4 to 8191 characters on mac os 10.6.8 (OpenSSL 1.0.0g 18 Jan 2012): Enter pass phrase for pk_1024.key.pem: 140735079038236:error:28069065:lib(40):UI_set_result:result too small:ui_lib.c:869:You must type in 4 to 1023 characters on both OS "Enter pass phrase for pk_1024.key.pem:" came prompt repeatedly end endless. Show quoted text

> perhaps try to change password to something longer then 3 chars by: > [testbox]$ openssl rsa -in pk_1024.key.pem -out pk_1024-new.key.pem -des

here I got same prompt bug. palik

Wed Feb 01 02:28:11 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 08:27:58 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

>> ad 1/ could you please check the encrypted private key (password is "123") >> [testbox]$ openssl rsa -in pk_1024.key.pem -text >>

> on squeeze: > openssl rsa -in pk_1024.key.pem -text > Enter pass phrase for pk_1024.key.pem: > 32373:error:28069065:lib(40):UI_set_result:result too small:ui_lib.c:850:You must type in 4 > to 8191 characters > > on mac os 10.6.8 (OpenSSL 1.0.0g 18 Jan 2012): > > Enter pass phrase for pk_1024.key.pem: > 140735079038236:error:28069065:lib(40):UI_set_result:result too small:ui_lib.c:869:You > must type in 4 to 1023 characters >

Hmm, for some reason default_passwd_cb seems to follow some openssl.conf option saying that minimum password len is 4 and does not allow the callback function to return password "123" I willl generate a new key with longer password + new tarball with cb-crash-test -- kmx

Wed Feb 01 02:33:19 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 08:33:08 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

FYI: I am trying to setup an old active perl - on http://www.activestate.com/activeperl/downloads "Access to older versions (such as Perl Perl 5.6, 5.8, or 5.10) is available in Business Edition <http://www.activestate.com/business-edition> and Enterprise Edition <http://www.activestate.com/enterprise-edition>." Perhaps I'll find some in my archive. -- kmx

Wed Feb 01 04:49:15 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 19:48:57 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, I have in my archive: ActivePerl-5.10.0.1002-MSWin32-x86-283697.msi ActivePerl-5.10.0.1004-MSWin32-x86-287188.msi ActivePerl-5.10.1.1007-MSWin32-x64-291969.msi ActivePerl-5.10.1.1007-MSWin32-x86-291969.msi ActivePerl-5.12.0.1200-MSWin32-x64-292396.msi ActivePerl-5.12.0.1200-MSWin32-x86-292396.msi ActivePerl-5.6.1.638-MSWin32-x86.msi ActivePerl-5.8.4.810-MSWin32-x86.msi ActivePerl-5.8.7.813-MSWin32-x86-148120.msi ActivePerl-5.8.8.817-MSWin32-x86-257965.msi ActivePerl-5.8.8.819-MSWin32-x86-267479.msi ActivePerl-5.8.8.820-MSWin32-x86-274739.msi ActivePerl-5.8.9.825-MSWin32-x86-288577.msi ActivePerl-5.8.9.827-MSWin32-x64-291969.msi all 15 to 20M each. Can email some if you want. On Wednesday, February 01, 2012 02:33:19 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > FYI: I am trying to setup an old active perl - on > http://www.activestate.com/activeperl/downloads > "Access to older versions (such as Perl Perl 5.6, 5.8, or 5.10) is > available in Business Edition > <http://www.activestate.com/business-edition> and Enterprise Edition > <http://www.activestate.com/enterprise-edition>." > > Perhaps I'll find some in my archive. > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Wed Feb 01 04:54:54 2012 info [...] maximka.de - Correspondence added

On Wed Feb 01 02:28:11 2012, kmx@volny.cz wrote: Show quoted text

>

> >> ad 1/ could you please check the encrypted private key (password is

> "123")

> >> [testbox]$ openssl rsa -in pk_1024.key.pem -text > >>

> > on squeeze: > > openssl rsa -in pk_1024.key.pem -text > > Enter pass phrase for pk_1024.key.pem: > > 32373:error:28069065:lib(40):UI_set_result:result too

> small:ui_lib.c:850:You must type in 4

> > to 8191 characters > > > > on mac os 10.6.8 (OpenSSL 1.0.0g 18 Jan 2012): > > > > Enter pass phrase for pk_1024.key.pem: > > 140735079038236:error:28069065:lib(40):UI_set_result:result too

> small:ui_lib.c:869:You

> > must type in 4 to 1023 characters > >

> > Hmm, for some reason default_passwd_cb seems to follow some > openssl.conf > option saying that minimum password len is 4 and does not allow the > callback function to return password "123"

I've created pk_1024-new.key.pem by using pk_1024-nopasswd.key.pem $ openssl rsa -in pk_1024-nopasswd.key.pem -out pk_1024-new.key.pem -des password: 1234 Both cb-crash-ng(2|4).pl died again by calling CTX_use_PrivateKey_file 'pk_1024- new.key.pem' failed palik

Subject:

pk_1024-new.key.pem

Download pk_1024-new.key.pem
application/octet-stream 958b

Message body not shown because it is not plain text.

Wed Feb 01 05:08:29 2012 kmx [...] volny.cz - Correspondence added

CC:	info [...] maximka.de, kmx [...] cpan.org
Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 11:08:19 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> I have a similar problem with cb-crash-ng2.pl and cb-crash-ng4.pl died at > CTX_use_PrivateKey_file call line 17 > > Problem is unknown algorithms. > > How did you generate the key files? >

Via my home made function based on calling: PEM_write_bio_PrivateKey(bp,pk,EVP_des_ede3_cbc(),passwd,passwd_len,cb,u); which is not working well. Show quoted text

> I wish to regenerate them here using algorithms that 0.9.8i understands. >

Sorry, I will change the keys. -- kmx

Wed Feb 01 05:19:15 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 11:19:03 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Hi, I have installed AFAIK the latest available ActivePerls from 5.8/5.10/5.12/5.14 series (I have used clean install) + made a fresh built of openssl 0.9.8e and 1.0.0g. To make test complete I have also used StrawberryPerl 5.14.2.1/32bit. I took updated the crash-test suite (see attached net-ssleay-callback-crash3.tar.gz) and run all tests with current svn trunk (rev288). The results are quite consistent: ActivePerl-5.14.2.1402-MSWin32-x86 + threads-1.85 + openssl-0.9.8e + vc6/cl 12.00.8804 - CRASH during cb-crash-ng4.pl ~ around thread:1248 - all the rest went fine ActivePerl-5.14.2.1402-MSWin32-x86 + threads-1.85 + openssl-1.0.0g + vc6/cl 12.00.8804 - CRASH during cb-crash-ng4.pl ~ around thread:6110 - all the rest went fine ActivePerl-5.12.4.1205-MSWin32-x86 + threads-1.82 + openssl-0.9.8e + vc6/cl 12.00.8804 - CRASH during cb-crash-ng4.pl ~ around thread:0245 - all the rest went fine ActivePerl-5.10.1.1007-MSWin32-x86 + threads-1.75 + openssl-0.9.8e + vc6/cl 12.00.8804 - CRASH during cb-crash-ng4.pl ~ around thread:0064 - all the rest went fine ActivePerl-5.8.9.827-MSWin32-x86 + threads-1.75 + openssl-0.9.8e + vc6/cl 12.00.8804 - CRASH during cb-crash-ng4.pl ~ around thread:2037 - all the rest went fine StrawberryPerl 5.14.2.1/32bit + threads-1.86 + openssl-1.0.0d + gcc-4.4.7-pre - cb-crash-ng3.pl RUNS VEEEEEERY LONG (this is CTX_new nearly-deadlock I have already reported) - cb-crash-ng3-nopasswd.pl RUNS VEEEEEERY LONG (dtto) - CRASH during cb-crash-ng4.pl ~ around thread:1702 - all the rest went fine Could you please run net-ssleay-callback-crash3.tar.gz on your UNIX-like boxes? (current svn trunk version) My conclusion - we have basically 2 remaining troubles: 1/ cross-thread callbacks are terribly broken (can be demonstrated by cb-crash-ng4.pl) 2/ CTX_new nearly-deadlock - which seems to be an issue only on Win32/gcc :( ad 1/ Making cross-thread callbacks possible would require quite huge changes, but with some effort it could be feasible - do we want to support cross-thread callbacks? (for me it is nearly-zero-priority) ad 2/ My openssl-locking-kung-fu patch fixes this issue but I understand that I was not able to bring a better crash case ("working" on more then 1 platform) to convince you. On the other hand openssl documentation says that thread-safe use of openssl lib requires to utilize locking functions and as far as I understand openssl-locking-kung-fu patch does not seem to have an adverse effect on any platform we have tested. (this is much higher priority for me) -- kmx

Download net-ssleay-callback-crash3.tar.gz
application/x-gzip 3.9k

Message body not shown because it is not plain text.

Wed Feb 01 05:19:16 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 20:19:02 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hello, My experience was that I could not read pk_1024.key.pem with 0.9.8, (though 1.0.0 was OK). Apprently due to teh encryption algorithm. Instead for those tests I use a Radiator test certificate (attached) it has the password 'whatever' Cheers. On Wednesday, February 01, 2012 04:54:54 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > On Wed Feb 01 02:28:11 2012, kmx@volny.cz wrote:

> > >> ad 1/ could you please check the encrypted private key (password > > >> is

> > > > "123") > >

> > >> [testbox]$ openssl rsa -in pk_1024.key.pem -text

> > > > > > on squeeze: > > > openssl rsa -in pk_1024.key.pem -text > > > Enter pass phrase for pk_1024.key.pem: > > > 32373:error:28069065:lib(40):UI_set_result:result too

> > > > small:ui_lib.c:850:You must type in 4 > >

> > > to 8191 characters > > > > > > on mac os 10.6.8 (OpenSSL 1.0.0g 18 Jan 2012): > > > > > > Enter pass phrase for pk_1024.key.pem: > > > 140735079038236:error:28069065:lib(40):UI_set_result:result too

> > > > small:ui_lib.c:869:You > >

> > > must type in 4 to 1023 characters

> > > > Hmm, for some reason default_passwd_cb seems to follow some > > openssl.conf > > option saying that minimum password len is 4 and does not allow the > > callback function to return password "123"

> > I've created pk_1024-new.key.pem by using pk_1024-nopasswd.key.pem > $ openssl rsa -in pk_1024-nopasswd.key.pem -out pk_1024-new.key.pem -des > password: 1234 > > Both cb-crash-ng(2|4).pl died again by calling CTX_use_PrivateKey_file > 'pk_1024- new.key.pem' failed > > > palik

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Download cert-srv.pem
application/x-pem-file 2.4k

Message body not shown because it is not plain text.

Wed Feb 01 05:29:58 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 11:29:48 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> all 15 to 20M each. Can email some if you want. >

5.6.1 please (if I understand correctly, we want to support Net::SSleay on this version) Related question - what is the oldest openssl version we want to support? in other words how old openssl makes sense to test with Net::SSLeay? -- kmx

Wed Feb 01 05:34:12 2012 info [...] maximka.de - Correspondence added

Show quoted text

> Instead for those tests I use a Radiator test certificate (attached) > it has > the password 'whatever'

I got again errors: perl cb-crash-ng2.pl --file cert-srv.pem Gonna start main thread part [thread:0000] Inside callback CTX_use_PrivateKey_file 'cert-srv.pem' failed at cb-crash-ng2.pl line 18. perl cb-crash-ng4.pl --file cert-srv.pem Gonna start main thread part [thread:0000] Inside callback CTX_use_PrivateKey_file 'cert-srv.pem' failed at cb-crash-ng4.pl line 20. openssl rsa -in cert-srv.pem -text | head Enter pass phrase for cert-srv.pem: writing RSA key Private-Key: (1024 bit) modulus: 00:d5:7c:35:95:fc:37:aa:a4:08:88:77:06:e5:2c: b4:83:1c:51:f9:69:6f:36:fa:35:ae:83:7d:59:fc: 50:85:ae:4d:6c:6c:c5:38:bd:46:c5:3e:12:34:aa: dd:4a:84:30:6a:a0:ee:49:d6:08:50:b4:63:6c:a7: ee:05:c4:aa:8e:fd:40:64:3b:6b:a3:a4:ea:92:10: 03:18:d4:e4:f5:0e:84:9a:be:d4:3a:78:26:37:ab: d4:f7:15:6b:d7:7a:28:68:0a:fc:cf:47:eb:80:98: 11:4c:65:89:82:27:c4:b6:d6:12:b4:52:22:16:53: palik

Wed Feb 01 06:00:43 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 20:57:32 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

On Wednesday, February 01, 2012 05:34:13 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >

> > Instead for those tests I use a Radiator test certificate (attached) > > it has > > the password 'whatever'

> > I got again errors: > > perl cb-crash-ng2.pl --file cert-srv.pem

Hmm, I dont think that syntax will work. I edited the file and made 2 changes: my $file = "$FindBin::Bin/cert-srv.pem"; and return "whatever"; # password Cheers. Show quoted text

> Gonna start main thread part > [thread:0000] Inside callback > CTX_use_PrivateKey_file 'cert-srv.pem' failed at cb-crash-ng2.pl line 18. > perl cb-crash-ng4.pl --file cert-srv.pem > Gonna start main thread part > [thread:0000] Inside callback > CTX_use_PrivateKey_file 'cert-srv.pem' failed at cb-crash-ng4.pl line 20. > > openssl rsa -in cert-srv.pem -text | head > Enter pass phrase for cert-srv.pem: > writing RSA key > Private-Key: (1024 bit) > modulus: > 00:d5:7c:35:95:fc:37:aa:a4:08:88:77:06:e5:2c: > b4:83:1c:51:f9:69:6f:36:fa:35:ae:83:7d:59:fc: > 50:85:ae:4d:6c:6c:c5:38:bd:46:c5:3e:12:34:aa: > dd:4a:84:30:6a:a0:ee:49:d6:08:50:b4:63:6c:a7: > ee:05:c4:aa:8e:fd:40:64:3b:6b:a3:a4:ea:92:10: > 03:18:d4:e4:f5:0e:84:9a:be:d4:3a:78:26:37:ab: > d4:f7:15:6b:d7:7a:28:68:0a:fc:cf:47:eb:80:98: > 11:4c:65:89:82:27:c4:b6:d6:12:b4:52:22:16:53: > > > palik

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Wed Feb 01 06:10:07 2012 info [...] maximka.de - Correspondence added

Show quoted text

> > perl cb-crash-ng2.pl --file cert-srv.pem

> > Hmm, I dont think that syntax will work. I edited the file and made 2 > changes:

pardon. I've forgotten to annotate my changes at both scripts: diff cb-crash-ng2.pl net-ssleay-callback-crash2/cb-crash-ng2.pl 7c7 < use Getopt::Long; --- Show quoted text

>

13,14c13 < GetOptions('file=s' => \$file); < -e $file || die "no file '$file'\n"; --- Show quoted text

>

18c17 < Net::SSLeay::CTX_use_PrivateKey_file($ctx, $file, &Net::SSLeay::FILETYPE_PEM) or die "CTX_use_PrivateKey_file '$file' failed"; --- Show quoted text

> Net::SSLeay::CTX_use_PrivateKey_file($ctx, $file, &Net::SSLeay::FILETYPE_PEM) or die

"CTX_use_PrivateKey_file failed"; diff cb-crash-ng4.pl net-ssleay-callback-crash2/cb-crash-ng4.pl 7d6 < use Getopt::Long; 14,15d12 < GetOptions('file=s' => \$file); < -e $file || die "no file '$file'\n"; 20c17 < Net::SSLeay::CTX_use_PrivateKey_file($ctx, $file, &Net::SSLeay::FILETYPE_PEM) or die "CTX_use_PrivateKey_file '$file' failed"; --- Show quoted text

> Net::SSLeay::CTX_use_PrivateKey_file($ctx, $file, &Net::SSLeay::FILETYPE_PEM) or die

"CTX_use_PrivateKey_file failed"; palik

Wed Feb 01 06:20:11 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 20:52:58 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

On Wednesday, February 01, 2012 05:29:59 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >

> > all 15 to 20M each. Can email some if you want.

> > 5.6.1 please

Attached Show quoted text

> (if I understand correctly, we want to support Net::SSleay > on this version)

Yes, I do. Show quoted text

> > Related question - what is the oldest openssl version we want to > support? in other words how old openssl makes sense to test with > Net::SSLeay?

There appears to be ifdefs relating to as far back as 9.5. So it might be fair to suggest that? Cheers. Show quoted text

> > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Download ActivePerl-5.6.1.638-MSWin32-x86.msi
application/x-msi 8.4m

Message body not shown because it is not plain text.

Wed Feb 01 06:20:25 2012 The RT System itself - System error

Sending the previous mail has failed. Please contact your admin, they can find more details in the logs.

Wed Feb 01 06:20:29 2012 The RT System itself - System error

Sending the previous mail has failed. Please contact your admin, they can find more details in the logs.

Wed Feb 01 06:36:06 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 21:24:34 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, On Wednesday, February 01, 2012 06:10:08 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >

> > > perl cb-crash-ng2.pl --file cert-srv.pem

> > > > Hmm, I dont think that syntax will work. I edited the file and made 2

>

> > changes:

> pardon. I've forgotten to annotate my changes at both scripts:

Oh, OK. You also need to change the decryption password to 'whatever' which is what was used for that certificate. Cheers. Show quoted text

> > diff cb-crash-ng2.pl net-ssleay-callback-crash2/cb-crash-ng2.pl > 7c7 > < use Getopt::Long; > --- > > 13,14c13 > < GetOptions('file=s' => \$file); > < -e $file || die "no file '$file'\n"; > --- > > 18c17 > < Net::SSLeay::CTX_use_PrivateKey_file($ctx, $file, > &Net::SSLeay::FILETYPE_PEM) or die "CTX_use_PrivateKey_file '$file' > failed"; > --- >

> > Net::SSLeay::CTX_use_PrivateKey_file($ctx, $file, > > &Net::SSLeay::FILETYPE_PEM) or die

> "CTX_use_PrivateKey_file failed"; > > > > > diff cb-crash-ng4.pl net-ssleay-callback-crash2/cb-crash-ng4.pl > 7d6 > < use Getopt::Long; > 14,15d12 > < GetOptions('file=s' => \$file); > < -e $file || die "no file '$file'\n"; > 20c17 > < Net::SSLeay::CTX_use_PrivateKey_file($ctx, $file, > &Net::SSLeay::FILETYPE_PEM) or die "CTX_use_PrivateKey_file '$file' > failed"; > --- >

> > Net::SSLeay::CTX_use_PrivateKey_file($ctx, $file, > > &Net::SSLeay::FILETYPE_PEM) or die

> "CTX_use_PrivateKey_file failed"; > > palik

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Wed Feb 01 06:55:24 2012 info [...] maximka.de - Correspondence added

Show quoted text

> Oh, OK. You also need to change the decryption password to 'whatever' > which is > what was used for that certificate.

You are right. My bad. Now it's work. palik

Wed Feb 01 07:00:27 2012 info [...] maximka.de - Correspondence added

I've tested net-ssleay-callback-crash3 again at squeeze perl v5.10.1 openssl 0.9.8o-4squeeze7 cb-crash-ng4.pl died by using svn-r288: perl cb-crash-ng4.pl 2&> net-ssleay-callback-crash3-result/svn-r288/cb-crash-ng4.log Segmentation fault palik On Wed Feb 01 05:19:15 2012, kmx@volny.cz wrote: Show quoted text

> Hi, > > I have installed AFAIK the latest available ActivePerls from > 5.8/5.10/5.12/5.14 series (I have used clean install) + made a fresh > built of openssl 0.9.8e and 1.0.0g. To make test complete I have also > used StrawberryPerl 5.14.2.1/32bit. > > I took updated the crash-test suite (see attached > net-ssleay-callback-crash3.tar.gz) and run all tests with current svn > trunk (rev288). > > The results are quite consistent: > > ActivePerl-5.14.2.1402-MSWin32-x86 + threads-1.85 + openssl-0.9.8e + > vc6/cl 12.00.8804 > - CRASH during cb-crash-ng4.pl ~ around thread:1248 > - all the rest went fine > > ActivePerl-5.14.2.1402-MSWin32-x86 + threads-1.85 + openssl-1.0.0g + > vc6/cl 12.00.8804 > - CRASH during cb-crash-ng4.pl ~ around thread:6110 > - all the rest went fine > > ActivePerl-5.12.4.1205-MSWin32-x86 + threads-1.82 + openssl-0.9.8e + > vc6/cl 12.00.8804 > - CRASH during cb-crash-ng4.pl ~ around thread:0245 > - all the rest went fine > > ActivePerl-5.10.1.1007-MSWin32-x86 + threads-1.75 + openssl-0.9.8e + > vc6/cl 12.00.8804 > - CRASH during cb-crash-ng4.pl ~ around thread:0064 > - all the rest went fine > > ActivePerl-5.8.9.827-MSWin32-x86 + threads-1.75 + openssl-0.9.8e + > vc6/cl 12.00.8804 > - CRASH during cb-crash-ng4.pl ~ around thread:2037 > - all the rest went fine > > StrawberryPerl 5.14.2.1/32bit + threads-1.86 + openssl-1.0.0d + > gcc-4.4.7-pre > - cb-crash-ng3.pl RUNS VEEEEEERY LONG (this is CTX_new nearly-deadlock I > have already reported) > - cb-crash-ng3-nopasswd.pl RUNS VEEEEEERY LONG (dtto) > - CRASH during cb-crash-ng4.pl ~ around thread:1702 > - all the rest went fine > > Could you please run net-ssleay-callback-crash3.tar.gz on your UNIX-like > boxes? (current svn trunk version) > > My conclusion - we have basically 2 remaining troubles: > 1/ cross-thread callbacks are terribly broken (can be demonstrated by > cb-crash-ng4.pl) > 2/ CTX_new nearly-deadlock - which seems to be an issue only on Win32/gcc :( > > ad 1/ Making cross-thread callbacks possible would require quite huge > changes, but with some effort it could be feasible - do we want to > support cross-thread callbacks? (for me it is nearly-zero-priority) > > ad 2/ My openssl-locking-kung-fu patch fixes this issue but I understand > that I was not able to bring a better crash case ("working" on more then > 1 platform) to convince you. On the other hand openssl documentation > says that thread-safe use of openssl lib requires to utilize locking > functions and as far as I understand openssl-locking-kung-fu patch does > not seem to have an adverse effect on any platform we have tested. (this > is much higher priority for me) > > -- > kmx

Subject:

net-ssleay-callback-crash3-result.tar.gz

Download net-ssleay-callback-crash3-result.tar.gz
application/x-gzip 1017.1k

Message body not shown because it is not plain text.

Wed Feb 01 07:25:53 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 22:17:38 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, Im seeing similar results on a range of platforms, though my tests are not complete. On Wednesday, February 01, 2012 05:19:17 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > Hi, > > I have installed AFAIK the latest available ActivePerls from > 5.8/5.10/5.12/5.14 series (I have used clean install) + made a fresh > built of openssl 0.9.8e and 1.0.0g. To make test complete I have also > used StrawberryPerl 5.14.2.1/32bit. > > I took updated the crash-test suite (see attached > net-ssleay-callback-crash3.tar.gz) and run all tests with current svn > trunk (rev288). > > The results are quite consistent: > > ActivePerl-5.14.2.1402-MSWin32-x86 + threads-1.85 + openssl-0.9.8e + > vc6/cl 12.00.8804 > - CRASH during cb-crash-ng4.pl ~ around thread:1248 > - all the rest went fine > > ActivePerl-5.14.2.1402-MSWin32-x86 + threads-1.85 + openssl-1.0.0g + > vc6/cl 12.00.8804 > - CRASH during cb-crash-ng4.pl ~ around thread:6110 > - all the rest went fine > > ActivePerl-5.12.4.1205-MSWin32-x86 + threads-1.82 + openssl-0.9.8e + > vc6/cl 12.00.8804 > - CRASH during cb-crash-ng4.pl ~ around thread:0245 > - all the rest went fine > > ActivePerl-5.10.1.1007-MSWin32-x86 + threads-1.75 + openssl-0.9.8e + > vc6/cl 12.00.8804 > - CRASH during cb-crash-ng4.pl ~ around thread:0064 > - all the rest went fine > > ActivePerl-5.8.9.827-MSWin32-x86 + threads-1.75 + openssl-0.9.8e + > vc6/cl 12.00.8804 > - CRASH during cb-crash-ng4.pl ~ around thread:2037 > - all the rest went fine > > StrawberryPerl 5.14.2.1/32bit + threads-1.86 + openssl-1.0.0d + > gcc-4.4.7-pre > - cb-crash-ng3.pl RUNS VEEEEEERY LONG (this is CTX_new nearly-deadlock I > have already reported) > - cb-crash-ng3-nopasswd.pl RUNS VEEEEEERY LONG (dtto) > - CRASH during cb-crash-ng4.pl ~ around thread:1702 > - all the rest went fine > > Could you please run net-ssleay-callback-crash3.tar.gz on your UNIX-like > boxes? (current svn trunk version) > > My conclusion - we have basically 2 remaining troubles: > 1/ cross-thread callbacks are terribly broken (can be demonstrated by > cb-crash-ng4.pl) > 2/ CTX_new nearly-deadlock - which seems to be an issue only on Win32/gcc :( > > ad 1/ Making cross-thread callbacks possible would require quite huge > changes, but with some effort it could be feasible - do we want to > support cross-thread callbacks? (for me it is nearly-zero-priority)

Do you mean: the callback occurs in another thread, not the one that established the callback? No I would not expect cross thread callbacks to be available in any case. This sort of thing is not likely to be well supported in perl either. Dont do it! Show quoted text

> > ad 2/ My openssl-locking-kung-fu patch fixes this issue but I understand > that I was not able to bring a better crash case ("working" on more then > 1 platform) to convince you. On the other hand openssl documentation > says that thread-safe use of openssl lib requires to utilize locking > functions and as far as I understand openssl-locking-kung-fu patch does > not seem to have an adverse effect on any platform we have tested. (this > is much higher priority for me)

Confusion: I dont think I have seen a patch called openssl-locking-kung-fu, or are you being facetious? I have been testing with fix-for-ctx_new-deadlock- v2_rev288.diff applied to 288. Correct? Is that what you would like to become the mainline? However, I dont think that on the platforms I have tested, that fix-for- ctx_new-deadlock-v2_rev288.diff makes any difference to the outcomes. So: are you convinced it really is necessary? Show quoted text

> > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Wed Feb 01 07:25:56 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 22:17:29 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Message body is not shown because it is too large.

Wed Feb 01 07:44:13 2012 kmx [...] volny.cz - Correspondence added

CC:	info [...] maximka.de, kmx [...] cpan.org
Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 13:44:04 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> Conclusion: all tests on all may easy platforms (with and without fix-for- > ctx_new-deadlock-v2_rev288.diff) worked fine except for cb-crash-ng4.pl which > failed on all platforms. >

Which means "cross-threaded" callbacks do not work, the rest is fine - for now it would be fine just to declare in documentation that it is not thread-safe practice to use cross-threads callbacks. Show quoted text

> Corollary: fix-for-ctx_new-deadlock-v2_rev288.diff made no difference on these > tests in these platforms. Does it really fix something thats broken? Evidence? >

The only broken thing I am able to demonstrate is nearly-deadlock when starting concurrently more threads trying to call CTX_new() - which is the test case cb-crash-ng3.pl (it nearly-hangs-up only on MSWin 32bit/gcc-compiled perl). Extra locking patch fixes this issue. We can: 1/ ignore it completely and mention potential CTX_new() deadlock in documentation 2/ try to find more crashy testcase 3/ make openssl locking initialization optional - e.g. Net::SSLeay::init_openssl_thread_safe_locking() 4/ turn on openssl locking automatically always when USE_ITHREADS (this does my patch) -- kmx

Wed Feb 01 08:29:43 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 14:29:34 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

>> all 15 to 20M each. Can email some if you want. >>

> 5.6.1 please (if I understand correctly, we want to support Net::SSleay > on this version) >

Do not send it, I have already found it on rapidshare -- kmx

Wed Feb 01 12:40:12 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 18:40:00 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> ... call CTX_new() - which is the test case cb-crash-ng3.pl (it > nearly-hangs-up only on MSWin 32bit/gcc-compiled perl).

You can grab a testing environment from http://strawberryperl.com/package/kmx/p5.14.2.1-RC/strawberry-perl-5.14.2.1-32bit-portable.zip (gcc and openssl are included; just use dmake instead of make the rest works like in any other perl environment) -- kmx

Wed Feb 01 14:41:28 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 20:41:17 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

see updated 61_threads-cb-crash.t (should work with older perls), IMHO ready to be included into test suite

Message body is not shown because sender requested not to inline it.

Wed Feb 01 15:05:38 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Wed, 01 Feb 2012 21:05:27 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

mike, ad openssl locking - to prove that it is not a virtual reality - untar openssl tarball - grep source codes for string CRYPTO_LOCK_ They really use it, not extensively but they do (I guess in 20-30 functions - just few of them we use from Net::SSleay). another deadlock/crash candidates: SSL_CTX_flush_sessions SSL_set_session SSL_SESSION_free SSL_CTX_add_session SSL_get1_session SSL_copy_session_id SSL_get_peer_certificate SSL_new -- kmx

Wed Feb 01 15:47:11 2012 info [...] maximka.de - Correspondence added

On Wed Feb 01 14:41:28 2012, kmx@volny.cz wrote: Show quoted text

> see updated 61_threads-cb-crash.t (should work with older perls), IMHO > ready to be included into test suite

on squeeze svn-r288+fix-for-ctx_new-deadlock-v2_rev288 t/local/61_threads-cb-crash.t .. ok palik

Wed Feb 01 15:50:00 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 02 Feb 2012 06:49:47 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

On Wednesday, February 01, 2012 07:44:15 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >

> > Conclusion: all tests on all may easy platforms (with and without > > fix-for- ctx_new-deadlock-v2_rev288.diff) worked fine except for > > cb-crash-ng4.pl which failed on all platforms.

> > Which means "cross-threaded" callbacks do not work, the rest is fine - > for now it would be fine just to declare in documentation that it is not > thread-safe practice to use cross-threads callbacks.

Agree. Show quoted text

>

> > Corollary: fix-for-ctx_new-deadlock-v2_rev288.diff made no difference on > > these tests in these platforms. Does it really fix something thats > > broken? Evidence?

> The only broken thing I am able to demonstrate is nearly-deadlock when > starting concurrently more threads trying to call CTX_new() - which is > the test case cb-crash-ng3.pl (it nearly-hangs-up only on MSWin > 32bit/gcc-compiled perl). Extra locking patch fixes this issue. > > We can: > 1/ ignore it completely and mention potential CTX_new() deadlock in > documentation > 2/ try to find more crashy testcase > 3/ make openssl locking initialization optional - e.g. > Net::SSLeay::init_openssl_thread_safe_locking() > 4/ turn on openssl locking automatically always when USE_ITHREADS (this > does my patch)

I think 4 is the preferred option. I you will send a final patch against 289 with updated doc and perhaps some portable test, I will apply it. Cheers. Show quoted text

> > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Wed Feb 01 16:00:43 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 02 Feb 2012 07:00:27 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, On Wednesday, February 01, 2012 02:41:29 PM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > see updated 61_threads-cb-crash.t (should work with older perls), IMHO > ready to be included into test suite

Not quite. Linux perl 5.14, openssl 1.0.0. gcc 4.6.2 on OpenSuSE 12.1 8 core Runs fine Windows Server 2003 32 bit activeperl 5.12, openssl 0.9.8i, MS Visual C 6, MS C compiler version 12 Takes about 30 secs to run, but OK. Windows Server 2003 32 bit activeperl 5.10, openssl 0.9.8i, MS Visual C 6, MS C compiler version 12 Takes about 30 secs to run, but OK. Windows Server 2003 32 bit activeperl 5.8.8, openssl 0.9.8i, MS Visual C 6, MS C compiler version 12 Takes about 30 secs, then message: A thread exited while 2 threads were running -- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Thu Feb 02 01:18:12 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 02 Feb 2012 07:17:59 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> Windows Server 2003 32 bit activeperl 5.8.8, openssl 0.9.8i, MS Visual C 6, > MS C compiler version 12 > Takes about 30 secs, then message: > A thread exited while 2 threads were running >

It is not as big trouble as it might seem. This happened due to my deadlock-detection-timer that simply does "hard exit" of the whole test after 30s. I have made this test less demanding - see attachment (it still reliably crashes on unpatched Net::SSLeay-1.42). -- kmx

Message body is not shown because sender requested not to inline it.

Thu Feb 02 01:19:45 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 02 Feb 2012 07:19:36 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> I you will send a final patch against 289 with updated doc and perhaps some > portable test, I will apply it. >

Thanks. I'll prepare patch+doc+test in release quality. -- kmx

Thu Feb 02 01:32:28 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 02 Feb 2012 16:32:15 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, Sorry, but the new version makes the same complaint on 5.8 Cheers. On Thursday, February 02, 2012 01:18:14 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >

> > Windows Server 2003 32 bit activeperl 5.8.8, openssl 0.9.8i, MS Visual > > C 6, MS C compiler version 12 > > Takes about 30 secs, then message: > > A thread exited while 2 threads were running

> > It is not as big trouble as it might seem. This happened due to my > deadlock-detection-timer that simply does "hard exit" of the whole test > after 30s. > > I have made this test less demanding - see attachment (it still reliably > crashes on unpatched Net::SSLeay-1.42). > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Thu Feb 02 03:05:01 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 02 Feb 2012 09:04:51 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

>> Which means "cross-threaded" callbacks do not work, the rest is fine - >> for now it would be fine just to declare in documentation that it is not >> thread-safe practice to use cross-threads callbacks. >>

> Agree. >

I have a way how to quite simply detect cross-thread callbacks (the action will be: throw a warning + do not call it). I will include this into my "hopefully-the-final-multi-threading-related-dark-magic-hack" I am working on. -- kmx

Fri Feb 03 07:07:00 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Fri, 03 Feb 2012 13:06:46 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Hi, please find my last-thread-safety-fix_r289.diff (xs code + doc) + 2 test files (*.t). It runs fine on all my platforms. The only troubles seems to be with pure 5.8.8 at it includes very old version of threads module (1.07) which quite hard to be sanely used. My test ends up like this: $ prove -bv t/local/61_threads-cb-crash.t t/local/61_threads-cb-crash....1..1 ok 1 - successfully finished, duration=0 A thread exited while 2 threads were running. ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.16 cusr + 0.00 csys = 0.16 CPU) Which is OK (the warning is caused by one detached thread that I intentionally do not want to finish, later versions of threads module does not warn about this). If you do not like the warning on 5.8.8 I propose skipping thread related tests if treads 1.71 (which corresponds to version released together with perl-5.8.9) or newer is not available. -- kmx

Message body is not shown because sender requested not to inline it.

Fri Feb 03 07:51:15 2012 info [...] maximka.de - Correspondence added

Hi, Mac OS X 10.6.8 perl v5.14.1 t/local/02_pod_coverage.t .............. skipped: currently disabled t/local/20_autoload.t .................. skipped: Some tests need Test::Exception t/local/30_error.t ..................... skipped: Requires Test::Exception, Test::Warn and Test::NoWarnings t/local/31_rsa_generate_key.t .......... skipped: Test::Exception required t/local/61_threads-cb-crash.t .......... ok t/local/62_threads-ctx_new-deadlock.t .. ok the rest ok too. What's wrong here? perl Makefile.PL said ... *** Found OpenSSL-0.9.8r installed in /usr but openssl version said OpenSSL 1.0.0g 18 Jan 2012 ------------------------ Ubuntu 10.04.3 perl v5.10.1 OpenSSL 0.9.8k 25 Mar 2009 t/local/02_pod_coverage.t .............. skipped: currently disabled t/local/30_error.t ..................... skipped: Requires Test::Exception, Test::Warn and Test::NoWarnings t/local/kwalitee.t ..................... skipped: Needs Test::Kwalitee t/local/61_threads-cb-crash.t .......... ok t/local/62_threads-ctx_new-deadlock.t .. ok the rest ok too ------------------------ Debian GNU/Linux 6.0 squeeze perl v5.10.1 OpenSSL 0.9.8o 01 Jun 2010 t/local/01_pod.t ....................... skipped: Test::Pod 1.00 required for testing POD t/local/02_pod_coverage.t .............. skipped: currently disabled t/local/20_autoload.t .................. ok t/local/30_error.t ..................... skipped: Requires Test::Exception, Test::Warn and Test::NoWarnings t/local/kwalitee.t ..................... skipped: Needs Test::Kwalitee t/local/61_threads-cb-crash.t .......... ok t/local/62_threads-ctx_new-deadlock.t .. ok the rest ok too palik

Fri Feb 03 18:01:57 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Sat, 04 Feb 2012 09:01:46 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, Thanks, I have rolled your patch in, but looks like svn is down at present. Will commit later today. On Friday, February 03, 2012 07:07:03 AM kmx via RT wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > Hi, > > please find my last-thread-safety-fix_r289.diff (xs code + doc) + 2 test > files (*.t). > > It runs fine on all my platforms. > > The only troubles seems to be with pure 5.8.8 at it includes very old > version of threads module (1.07) which quite hard to be sanely used. > > My test ends up like this: > > $ prove -bv t/local/61_threads-cb-crash.t > t/local/61_threads-cb-crash....1..1 > ok 1 - successfully finished, duration=0 > A thread exited while 2 threads were running. > ok > All tests successful. > Files=1, Tests=1, 0 wallclock secs ( 0.16 cusr + 0.00 csys = 0.16 CPU) > > Which is OK (the warning is caused by one detached thread that I > intentionally do not want to finish, later versions of threads module > does not warn about this). > > If you do not like the warning on 5.8.8 I propose skipping thread > related tests if treads 1.71 (which corresponds to version released > together with perl-5.8.9) or newer is not available. > > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Fri Feb 03 19:13:23 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Sat, 04 Feb 2012 10:13:13 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

OK, Committed 290 Cheers. On Saturday, February 04, 2012 09:01:46 AM you wrote: Show quoted text

> Hi, > > Thanks, I have rolled your patch in, but looks like svn is down at present. > Will commit later today. > > On Friday, February 03, 2012 07:07:03 AM kmx via RT wrote:

> > Queue: Net-SSLeay > > > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > > > Hi, > > > > please find my last-thread-safety-fix_r289.diff (xs code + doc) + 2 test > > files (*.t). > > > > It runs fine on all my platforms. > > > > The only troubles seems to be with pure 5.8.8 at it includes very old > > version of threads module (1.07) which quite hard to be sanely used. > > > > My test ends up like this: > > > > $ prove -bv t/local/61_threads-cb-crash.t > > t/local/61_threads-cb-crash....1..1 > > ok 1 - successfully finished, duration=0 > > A thread exited while 2 threads were running. > > ok > > All tests successful. > > Files=1, Tests=1, 0 wallclock secs ( 0.16 cusr + 0.00 csys = 0.16 > > CPU) > > > > Which is OK (the warning is caused by one detached thread that I > > intentionally do not want to finish, later versions of threads module > > does not warn about this). > > > > If you do not like the warning on 5.8.8 I propose skipping thread > > related tests if treads 1.71 (which corresponds to version released > > together with perl-5.8.9) or newer is not available. > > > > -- > > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Sun Feb 05 16:01:22 2012 kmx [...] cpan.org - Correspondence added

Show quoted text

> OK, Committed 290

Cool, thanks men.

Sun Feb 05 17:49:22 2012 MIKEM [...] cpan.org - TimeWorked changed from (no value) to '600'

Sun Feb 05 17:49:22 2012 MIKEM [...] cpan.org - Status changed from 'open' to 'resolved'

Mon Feb 06 04:10:46 2012 info [...] maximka.de - Correspondence added

thank you guys for fast bug solving. regards, palik

Mon Feb 06 04:10:52 2012 The RT System itself - Status changed from 'resolved' to 'open'

Sun Mar 18 19:25:52 2012 MIKEM [...] cpan.org - Correspondence added

kmx, have you seen subsequent discussion of this patch and resulting segfault at: https://rt.cpan.org/Public/Bug/Display.html?id=75841 I would value your input on that. Cheers. On Mon Feb 06 04:10:46 2012, palik wrote: Show quoted text

> thank you guys for fast bug solving. > > regards, > palik

Thu Mar 22 02:22:46 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 22 Mar 2012 07:22:32 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

I am sending a cosmetic changes related to threads. It turned out that OPENSSL_THREADS we use in #ifdefs is defined only in openssl-0.9.7 and newer. Therefore I have simplified related #ifdefs and officially declare threading support since 0.9.7 -- kmx

Message body is not shown because sender requested not to inline it.

Thu Mar 22 03:19:10 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 22 Mar 2012 17:19:03 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, On Thursday, March 22, 2012 02:22:48 AM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > I am sending a cosmetic changes related to threads. > > It turned out that OPENSSL_THREADS we use in #ifdefs is defined only in > openssl-0.9.7 and newer. > > Therefore I have simplified related #ifdefs and officially declare > threading support since 0.9.7

Thanks. tests fine here. Committed at 324 Cheers. Show quoted text

> > -- > kmx

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Thu Mar 22 06:35:00 2012 info [...] maximka.de - Correspondence added

Hi, no idea if you expect test on my part. How ever I've tested r324 too. 1. wheezy, openssl 1.0.0h-1, perl v5.14.2 ... t/local/07_sslecho.t ................... DEBUG: BOOT start DEBUG: init - begin DEBUG: setting static locking DEBUG: setting dynamic locking DEBUG: init - end DEBUG: BOOT done: tid=0 my_perl=0 Not enough arguments for Net::SSLeay::get_shared_ciphers at t/local/07_sslecho.t line 64, near "$ssl)" Execution of t/local/07_sslecho.t aborted due to compilation errors. # Looks like your test exited with 255 before it could output anything. t/local/07_sslecho.t ................... Dubious, test returned 255 (wstat 65280, 0xff00) Failed 78/78 subtests ... Test Summary Report ------------------- t/local/07_sslecho.t (Wstat: 65280 Tests: 0 Failed: 0) Non-zero exit status: 255 Parse errors: Bad plan. You planned 78 tests but ran 0. Files=24, Tests=1617, 4 wallclock secs ( 0.14 usr 0.41 sys + 1.53 cusr 1.44 csys = 3.52 CPU) Result: FAIL 2. MAC OS X 10.6.8, opensslL 1.0.0g, perl v5.14.1 t/local/33_x509_create_cert.t .......... 1/124 # Failed test 'ASN1_INTEGER_get' # at t/local/33_x509_create_cert.t line 52. # got: '123456789123456789' # expected: '-1' t/local/33_x509_create_cert.t .......... 46/124 # Looks like you failed 1 test of 124. t/local/33_x509_create_cert.t .......... Dubious, test returned 1 (wstat 256, 0x100) Failed 1/124 subtests t/local/34_x509_crl.t .... ... Test Summary Report ------------------- t/local/33_x509_create_cert.t (Wstat: 256 Tests: 124 Failed: 1) Failed test: 21 Non-zero exit status: 1 Files=24, Tests=2130, 5 wallclock secs ( 0.32 usr 0.10 sys + 2.69 cusr 0.30 csys = 3.41 CPU) Result: FAIL palik On Thu Mar 22 03:19:10 2012, mikem@open.com.au wrote: Show quoted text

> Hi, > > > On Thursday, March 22, 2012 02:22:48 AM you wrote:

> > Queue: Net-SSLeay > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > > > I am sending a cosmetic changes related to threads. > > > > It turned out that OPENSSL_THREADS we use in #ifdefs is defined only in > > openssl-0.9.7 and newer. > > > > Therefore I have simplified related #ifdefs and officially declare > > threading support since 0.9.7

> > Thanks. tests fine here. > > Committed at 324 > Cheers. >

> > > > -- > > kmx

Thu Mar 22 07:28:15 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 22 Mar 2012 12:28:05 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> no idea if you expect test on my part. How ever I've tested r324 too. > 1. wheezy, openssl 1.0.0h-1, perl v5.14.2 > ... > t/local/07_sslecho.t ................... DEBUG: BOOT start > DEBUG: init - begin > DEBUG: setting static locking > DEBUG: setting dynamic locking > DEBUG: init - end > DEBUG: BOOT done: tid=0 my_perl=0 > Not enough arguments for Net::SSLeay::get_shared_ciphers at t/local/07_sslecho.t line 64, > near "$ssl)" >

Please try to test on a clean svn checkout - this should not be failing Show quoted text

> 2. MAC OS X 10.6.8, opensslL 1.0.0g, perl v5.14.1 > > t/local/33_x509_create_cert.t .......... 1/124 > # Failed test 'ASN1_INTEGER_get' > # at t/local/33_x509_create_cert.t line 52. > # got: '123456789123456789' > # expected: '-1' > t/local/33_x509_create_cert.t .......... 46/124 # Looks like you failed 1 test of 124. > t/local/33_x509_create_cert.t .......... Dubious, test returned 1 (wstat 256, 0x100) >

This is a bug in test suite - your MacOSX has probably 'long int' that is longer than usual :) '123456789123456789' is correct serial number; however the test expects -1 as the serial number is longer than 32bit integer Could you please create a separate RT for this test failure. -- kmx

Thu Mar 22 07:56:14 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 22 Mar 2012 12:56:03 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	kmx <kmx [...] volny.cz>

Show quoted text

> t/local/33_x509_create_cert.t .......... 1/124 > # Failed test 'ASN1_INTEGER_get' > # at t/local/33_x509_create_cert.t line 52. > # got: '123456789123456789' > # expected: '-1' >

Try enclosed serial_number_fix.diff

Message body is not shown because sender requested not to inline it.

Thu Mar 22 08:23:03 2012 info [...] maximka.de - Correspondence added

Show quoted text

> t/local/07_sslecho.t line 64,

> > near "$ssl)"

> Please try to test on a clean svn checkout - this should not be > failing

r327 prove t/local/ ... t/local/07_sslecho.t ................... ok ... All tests successful. Files=25, Tests=1714, 4 wallclock secs ( 0.14 usr 0.42 sys + 1.56 cusr 1.60 csys = 3.72 CPU) Result: PASS palik On Thu Mar 22 07:28:15 2012, kmx@volny.cz wrote: Show quoted text

>

> > no idea if you expect test on my part. How ever I've tested r324

> too.

> > 1. wheezy, openssl 1.0.0h-1, perl v5.14.2 > > ... > > t/local/07_sslecho.t ................... DEBUG: BOOT start > > DEBUG: init - begin > > DEBUG: setting static locking > > DEBUG: setting dynamic locking > > DEBUG: init - end > > DEBUG: BOOT done: tid=0 my_perl=0 > > Not enough arguments for Net::SSLeay::get_shared_ciphers at

> t/local/07_sslecho.t line 64,

> > near "$ssl)" > >

> > Please try to test on a clean svn checkout - this should not be > failing > >

> > 2. MAC OS X 10.6.8, opensslL 1.0.0g, perl v5.14.1 > > > > t/local/33_x509_create_cert.t .......... 1/124 > > # Failed test 'ASN1_INTEGER_get' > > # at t/local/33_x509_create_cert.t line 52. > > # got: '123456789123456789' > > # expected: '-1' > > t/local/33_x509_create_cert.t .......... 46/124 # Looks like you

> failed 1 test of 124.

> > t/local/33_x509_create_cert.t .......... Dubious, test returned 1

> (wstat 256, 0x100)

> >

> > This is a bug in test suite - your MacOSX has probably 'long int' that > is longer than usual :) > > '123456789123456789' is correct serial number; however the test > expects > -1 as the serial number is longer than 32bit integer > > Could you please create a separate RT for this test failure. > > -- > kmx

Thu Mar 22 16:51:54 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Fri, 23 Mar 2012 06:51:51 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, Tests OK here. Do you want me to commit? Cheers. On Thursday, March 22, 2012 07:56:16 AM kmx via RT wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >

> > t/local/33_x509_create_cert.t .......... 1/124 > > # Failed test 'ASN1_INTEGER_get' > > # at t/local/33_x509_create_cert.t line 52. > > # got: '123456789123456789' > > # expected: '-1'

> > Try enclosed serial_number_fix.diff

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Thu Mar 22 17:48:42 2012 kmx [...] volny.cz - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Thu, 22 Mar 2012 22:48:19 +0100
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Kmx <kmx [...] volny.cz>

Yes please, commit "mikem@open.com.au via RT" <bug-Net-SSLeay@rt.cpan.org>napsal/a: Show quoted text

><URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > >Hi, > >Tests OK here. >Do you want me to commit? > >Cheers. > >On Thursday, March 22, 2012 07:56:16 AM kmx via RT wrote:

>> Queue: Net-SSLeay >> Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > >>

>> > t/local/33_x509_create_cert.t .......... 1/124 >> > # Failed test 'ASN1_INTEGER_get' >> > # at t/local/33_x509_create_cert.t line 52. >> > # got: '123456789123456789' >> > # expected: '-1'

>> >> Try enclosed serial_number_fix.diff

>-- >Mike McCauley mikem@open.com.au >Open System Consultants Pty. Ltd >9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au >Phone +61 7 5598-7474 Fax +61 7 5598-7070 > >Radiator: the most portable, flexible and configurable RADIUS server >anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, >Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, >TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, >DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc. > >

Thu Mar 22 18:31:09 2012 kmx [...] cpan.org - Correspondence added

On Thu Mar 22 17:48:42 2012, kmx@volny.cz wrote:
Show quoted text

> Yes please, commit

and close this RT

Thu Mar 22 19:15:46 2012 mikem [...] open.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #73532] using Net::SSLeay in threads
Date:	Fri, 23 Mar 2012 09:15:40 +1000
To:	bug-Net-SSLeay [...] rt.cpan.org
From:	Mike McCauley <mikem [...] open.com.au>

Hi, On Thursday, March 22, 2012 05:48:43 PM you wrote: Show quoted text

> Queue: Net-SSLeay > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > Yes please, commit

Done at 328 Cheers. Show quoted text

> > "mikem@open.com.au via RT" <bug-Net-SSLeay@rt.cpan.org>napsal/a:

> ><URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > > > >Hi, > > > >Tests OK here. > >Do you want me to commit? > > > >Cheers. > > > >On Thursday, March 22, 2012 07:56:16 AM kmx via RT wrote:

> >> Queue: Net-SSLeay > >> > >> Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=73532 > > >>

> >> > t/local/33_x509_create_cert.t .......... 1/124 > >> > # Failed test 'ASN1_INTEGER_get' > >> > # at t/local/33_x509_create_cert.t line 52. > >> > # got: '123456789123456789' > >> > # expected: '-1'

> >> > >> Try enclosed serial_number_fix.diff

-- Mike McCauley mikem@open.com.au Open System Consultants Pty. Ltd 9 Bulbul Place Currumbin Waters QLD 4223 Australia http://www.open.com.au Phone +61 7 5598-7474 Fax +61 7 5598-7070 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

Thu Mar 22 19:23:31 2012 MIKEM [...] cpan.org - Status changed from 'open' to 'resolved'