Skip Menu |

This queue is for tickets about the threads CPAN distribution.

Report information
The Basics
Id: 34203
Status: resolved
Priority: 0/
Queue: threads
People
Owner: Nobody in particular
Requestors: oliver.cook [...] betfair.com
Cc:
AdminCc:
Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)

History Show all quoted text
  Tue Mar 18 15:02:11 2008 oliver.cook [...] betfair.com - Ticket created
Subject: glibc errors when using LWP and threads
Date: Tue, 18 Mar 2008 19:00:13 -0000
To: <bug-threads [...] rt.cpan.org>, <bug-libwww-perl [...] rt.cpan.org>
From: "Oliver Cook" <oliver.cook [...] betfair.com>
Hi, Whilst developing a multithreaded PERL application using the SOAP::Lite and threads modules, I encountered memory allocation errors such as the following resulting in PERL aborting: *** glibc detected *** corrupted double-linked list: 0x0000002a99400098 *** *** glibc detected *** double free or corruption (fasttop): 0x0000002af33af880 *** *** glibc detected *** free(): invalid pointer: 0x0000002aa0e494d0 *** I've been able to reproduce the problem on a number of architectures running different versions of PERL: * PERL 5.8.6 * PERL 5.8.8 * PERL 5.10.0 * x86_64 and i686 architectures * Intel P4 with HyperThreading ('dual' core) * Dual Socket, Dual Core AMD Opteron 280 (quad core) Most recently my testing has been on PERL 5.8.8, using LWP 5.808 and threads 1.69. I've been able to reduce the problem to a simple test case that doesn't use SOAP::Lite but only LWP, so SOAP::Lite is off the hook! The resultant 30-line script that causes the failure can be downloaded from http://www.olliecook.net/glibc-perl-threads-LWP.pl It results in output such as: # ./glibc-perl-threads-LWP.pl Demonstration of glibc error using LWP version 5.808 and PERL 5.8.8 *** glibc detected *** double free or corruption (fasttop): 0x0000002ac1bbda40 *** Aborted Is there any further debugging I can do myself in order to determine whether the cause of the problem is in PERL itself or either of the LWP or threads modules? Furthermore, if there's any further information I can provide that would help please don't hesitate to ask. Thanks, Ollie --->8--->8--->8 perl -V output --->8--->8--->8 # perl -V Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=linux, osvers=2.6.9-42.0.3.elsmp, archname=x86_64-linux-thread-multi uname='linux builder6.centos.org 2.6.9-42.0.3.elsmp #1 smp fri oct 6 06:28:26 cdt 2006 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Doptimize=-O2 -g -pipe -m64 -Dversion=5.8.8 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Dprivlib=/usr/lib/perl5/5.8.8 -Dsitelib=/usr/lib/perl5/site_perl/5.8.8 -Dvendorlib=/usr/lib/perl5/vendor_perl/5.8.8 -Darchlib=/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi -Dsitearch=/usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi -Dvendorarch=/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-mult i -Darchname=x86_64-linux -Dvendorprefix=/usr -Dsiteprefix=/usr -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dinc_version_list=5.8.7 5.8.6 5.8.5 5.8.4 5.8.3 -Dscriptdir=/usr/bin' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm', optimize='-O2 -g -pipe -m64', cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -I/usr/include/gdbm' ccversion='', gccversion='3.4.6 20060404 (Red Hat 3.4.6-8)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='gcc', ldflags ='' libpth=/usr/local/lib64 /lib64 /usr/lib64 libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc libc=, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.3.4' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE' cccdlflags='-fPIC', lddlflags='-shared' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_REENTRANT_API Built under linux Compiled at Nov 5 2007 14:18:42 @INC: /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi /usr/lib64/perl5/site_perl/5.8.7/x86_64-linux-thread-multi /usr/lib64/perl5/site_perl/5.8.6/x86_64-linux-thread-multi /usr/lib64/perl5/site_perl/5.8.5/x86_64-linux-thread-multi /usr/lib64/perl5/site_perl/5.8.4/x86_64-linux-thread-multi /usr/lib64/perl5/site_perl/5.8.3/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.5 /usr/lib/perl5/site_perl/5.8.4 /usr/lib/perl5/site_perl/5.8.3 /usr/lib/perl5/site_perl /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.8.7/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.8.6/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.8.5/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.8.4/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.8.3/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl/5.8.5 /usr/lib/perl5/vendor_perl/5.8.4 /usr/lib/perl5/vendor_perl/5.8.3 /usr/lib/perl5/vendor_perl /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi /usr/lib/perl5/5.8.8 . # uname -a Linux xyz 2.6.9-67.0.7.ELsmp #1 SMP Sat Mar 15 06:33:45 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux Ollie Cook Performance and Capacity Office: +44 20 8834 8128 Yahoo! Messenger: olliebetfair Betfair Limited | Winslow Road | Hammersmith Embankment | London | W6 9HP Company No. 5140986 The information in this e-mail and any attachment is confidential and is intended only for the named recipient(s). The e-mail may not be disclosed or used by any person other than the addressee, nor may it be copied in any way. If you are not a named recipient please notify the sender immediately and delete any copies of this message. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any view or opinions presented are solely those of the author and do not necessarily represent those of the company. Show quoted text
________________________________________________________________________ In order to protect our email recipients, Betfair Group use SkyScan from MessageLabs to scan all Incoming and Outgoing mail for viruses.
________________________________________________________________________
  Mon Mar 24 15:39:48 2008 jdhedden [...] cpan.org - Correspondence added
From: jdhedden [...] cpan.org
oliver.cook@betfair.com wrote: Show quoted text
> I encountered memory allocation errors such as the > following resulting in PERL aborting: > > *** glibc detected *** corrupted double-linked list: > 0x0000002a99400098 *** > *** glibc detected *** double free or corruption (fasttop): > 0x0000002af33af880 *** > *** glibc detected *** free(): invalid pointer: 0x0000002aa0e494d0 *** > > A 30-line script that causes the failure can be downloaded > from http://www.olliecook.net/glibc-perl-threads-LWP.pl
This bug appears to be the same as: http://rt.perl.org/rt3/Public/Bug/Display.html?id=45053 My reasoning is that both test scripts involve loading IO (LWP does cause IO to be loaded) simultaneously inside many threads. This bug appears to have been fixed by changes 32091, 33492 and 33498 to blead: http://www.mail-archive.com/perl5-changes@perl.org/msg18711.html http://www.mail-archive.com/perl5-changes@perl.org/msg20078.html http://www.mail-archive.com/perl5-changes@perl.org/msg20084.html I tried the test script under the following and could not reproduce the failure: blead @ 33550 under Solaris blead @ 33550 under Cygwin 5.10.x @ 33549 under Cygwin (I could reproduce the failure with 5.8.8 under Solaris.)
  Mon Mar 24 15:39:53 2008 The RT System itself - Status changed from 'new' to 'open'
  Mon Mar 24 15:42:19 2008 jdhedden [...] cpan.org - Correspondence added
One workaround is to ensure that all modules are loaded in main thread before creating child threads. This can be done using an "LWP fetch" in the main thread.
  Mon Mar 24 15:42:29 2008 jdhedden [...] cpan.org - Status changed from 'open' to 'resolved'