Bug #63083 for PAR-Packer: pp: Undefined subroutine &DynaLoader::bootstrap called in pp-compiled bianry

Wed Nov 17 08:13:18 2010 cstef [...] parallel.ru - Ticket created

Subject:

pp: Undefined subroutine &DynaLoader::bootstrap called in pp-compiled bianry

I compile simple file #!/usr/bin/perl use IO (); with pp -o a.exe a.pl On the same machine it works OK. Then I transfer it to another machine and get: $ ./a.exe Undefined subroutine &DynaLoader::bootstrap called at /usr/lib/perl5/x86_64-linux/XSLoader.pm line 105. Compilation failed in require at /usr/lib/perl5/x86_64-linux/IO/Handle.pm line 263. BEGIN failed--compilation aborted at /usr/lib/perl5/x86_64-linux/IO/Handle.pm line 263. Compilation failed in require at /usr/lib/perl5/x86_64-linux/IO/Seekable.pm line 101. BEGIN failed--compilation aborted at /usr/lib/perl5/x86_64-linux/IO/Seekable.pm line 101. Compilation failed in require at /usr/lib/perl5/x86_64-linux/IO/File.pm line 133. BEGIN failed--compilation aborted at /usr/lib/perl5/x86_64-linux/IO/File.pm line 133. Compilation failed in require at -e line 351. Some details about machines: Compile machine: $ pp --version PAR Packager, version 1.007 (PAR version 1.002) $ perl -v This is perl, v5.8.8 built for x86_64-linux-thread-multi $ uname -a Linux t60-2.parallel.ru 2.6.18-skif-rhel-alt13.M41.3 #1 SMP Tue Feb 2 12:09:59 MSK 2010 x86_64 GNU/Linux Target machine (with binary not working): $ perl -v This is perl, v5.8.9 built for x86_64-linux-thread-multi $ uname -a Linux access.lomonosov.parallel.ru 2.6.27-tmc-srv-tmc76 #1 SMP Fri Aug 27 20:32:25 UTC 2010 x86_64 GNU/Linux How can I make binary using IO that can be transferred to another machine?

Wed Nov 17 08:56:17 2010 RSCHUPP [...] cpan.org - Correspondence added

On 2010-11-17 08:13:18, Coctic wrote: Show quoted text

> Then I transfer it to another machine and get: > $ ./a.exe > Undefined subroutine &DynaLoader::bootstrap called at > /usr/lib/perl5/x86_64-linux/XSLoader.pm line 105.

That's not supposed to happen :( A couple of questions: - did you build PAR::Packer yourself? - if yes - did you build it on the machine where you do the packing? - did you run "make test" and all tests succeeded? Can you run the following on the machine where the packed executable fails: strace -o a.strace -f -v -e trace=file,process ./a.exe gzip the resulting a.strace and attach it to the ticket. Cheers, Roderich

Wed Nov 17 08:56:18 2010 The RT System itself - Status changed from 'new' to 'open'

Wed Nov 17 09:26:36 2010 cstef [...] parallel.ru - Correspondence added

From:

cstef [...] parallel.ru

On Wed Nov 17 08:56:17 2010, RSCHUPP wrote: Show quoted text

> - did you build PAR::Packer yourself? > - if yes > - did you build it on the machine where you do the packing? > - did you run "make test" and all tests succeeded?

I ran cpan shell, then used 'install pp' command. As I remeber, it did the tests and did not complain about it. I can try re-make the tests if it's needed. Show quoted text

> gzip the resulting a.strace and attach it to the ticket.

Done it.

Subject:

a.strace.gz

Download a.strace.gz
application/binary 2.8k

Message body not shown because it is not plain text.

Wed Nov 17 10:30:16 2010 cstef [...] parallel.ru - Correspondence added

From:

cstef [...] parallel.ru

Show quoted text

> - did you build PAR::Packer yourself? > - if yes > - did you build it on the machine where you do the packing?

Forgot to say: yes, it was done on the same machine that did the packing. Show quoted text

> - did you run "make test" and all tests succeeded?

I rebuilt pp, cpan skipped 1 test, all others succeeded. The bug did not go away.

Wed Nov 17 12:28:26 2010 RSCHUPP [...] cpan.org - Correspondence added

Thanks for info and strace. strace shows that a.exe pulled in /usr/lib64/libperl.so.5.8, i.e the shared perl library from the system where it ran (instead of the copy that is packed in the executable). Since this is from another version of Perl it might explain the problem. But it isn't supposed to do that... strace also shows that you ran a.exe with LD_LIBRARY_PATH set in the environment. Can you rerun a.exe without it and see if the problem goes away? Another thing to try: first remove PAR's cache area which according to strace was /home/users/cstef/tmp/par-cstef (you have TMPDIR=$HOME/tmp, right?) then run a.exe and look for a file also called a.exe in the cache area, should be something like /home/users/cstef/tmp/par-cstef/cache-LONGSHA1DIGEST/a.exe and post the output of readelf -d /home/users/cstef/tmp/par-cstef/cache-LONGSHA1DIGEST/a.exe In the same directory there should also be a file matching libperl*so* -what is its name? Cheers, Roderich

Thu Nov 18 01:51:24 2010 cstef [...] parallel.ru - Correspondence added

From:

cstef [...] parallel.ru

On Wed Nov 17 12:28:26 2010, RSCHUPP wrote: Show quoted text

> strace also shows that you ran a.exe with LD_LIBRARY_PATH set > in the environment. Can you rerun a.exe without it and see > if the problem goes away?

No, it doesn't: $ export LD_LIBRARY_PATH="" cstef@access:~>> $ ./a.exe Undefined subroutine &DynaLoader::bootstrap called at /usr/lib/perl5/x86_64-linux/XSLoader.pm line 105. Compilation failed in require at /usr/lib/perl5/x86_64-linux/IO/Handle.pm line 263. BEGIN failed--compilation aborted at /usr/lib/perl5/x86_64-linux/IO/Handle.pm line 263. Compilation failed in require at /usr/lib/perl5/x86_64-linux/IO/Seekable.pm line 101. BEGIN failed--compilation aborted at /usr/lib/perl5/x86_64-linux/IO/Seekable.pm line 101. Compilation failed in require at /usr/lib/perl5/x86_64-linux/IO/File.pm line 133. BEGIN failed--compilation aborted at /usr/lib/perl5/x86_64-linux/IO/File.pm line 133. Compilation failed in require at -e line 351. Show quoted text

> Another thing to try: first remove PAR's cache area > which according to strace was > > /home/users/cstef/tmp/par-cstef > > (you have TMPDIR=$HOME/tmp, right?)

Right. Show quoted text

> then run a.exe and look for a file also called a.exe in the

Show quoted text

> readelf -d /home/users/cstef/tmp/par-cstef/cache-LONGSHA1DIGEST/a.exe

$ readelf -d /home/users/cstef/tmp/par-cstef/cache-91983bfecf21472d244f6bf26116bc20ee4433d3/a.exe Dynamic section at offset 0xc028 contains 22 entries: Tag Type Name/Value 0x0000000000000001 (NEEDED) Shared library: [libperl.so.5.8] 0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] 0x000000000000000c (INIT) 0x401dd0 0x000000000000000d (FINI) 0x405b38 0x000000006ffffef5 (GNU_HASH) 0x400240 0x0000000000000005 (STRTAB) 0x400ea8 0x0000000000000006 (SYMTAB) 0x4003e0 0x000000000000000a (STRSZ) 1501 (bytes) 0x000000000000000b (SYMENT) 24 (bytes) 0x0000000000000015 (DEBUG) 0x0 0x0000000000000003 (PLTGOT) 0x60c1e0 0x0000000000000002 (PLTRELSZ) 1896 (bytes) 0x0000000000000014 (PLTREL) RELA 0x0000000000000017 (JMPREL) 0x401668 0x0000000000000007 (RELA) 0x401620 0x0000000000000008 (RELASZ) 72 (bytes) 0x0000000000000009 (RELAENT) 24 (bytes) 0x000000006ffffffe (VERNEED) 0x401570 0x000000006fffffff (VERNEEDNUM) 3 0x000000006ffffff0 (VERSYM) 0x401486 0x0000000000000000 (NULL) 0x0 Show quoted text

> In the same directory there should also be a file matching libperl*so* > -what is its name?

$ ls /home/users/cstef/tmp/par-cstef/cache-91983bfecf21472d244f6bf26116bc20ee4433d3/libperl* /home/users/cstef/tmp/par-cstef/cache-91983bfecf21472d244f6bf26116bc20ee4433d3/libperl.so.5.8.8

Thu Nov 18 02:05:40 2010 cstef [...] parallel.ru - Correspondence added

From:

cstef [...] parallel.ru

On Wed Nov 17 12:28:26 2010, RSCHUPP wrote: Show quoted text

> strace also shows that you ran a.exe with LD_LIBRARY_PATH set > in the environment. Can you rerun a.exe without it and see > if the problem goes away?

Simply setting LD_LIBRARY_PATH to "" does not solve the problem, but using LD_PRELOAD="/home/users/cstef/tmp/par-cstef/cache-91983bfecf21472d244f6bf26116bc20ee4433d3/libperl.so.5.8.8" ./a.exe does - a.exe now works (and my binary, which I was trying to run initially, works too, a.exe was only proof-of-concept). So workaround is found, thank you for help. Well, if it will be fixed in pp distribution, it would be very nice. It some more information is needed, I am ready.

Thu Nov 18 04:38:06 2010 RSCHUPP [...] cpan.org - Correspondence added

On 2010-11-18 01:51:24, Coctic wrote: Show quoted text

> $ readelf -d >

/home/users/cstef/tmp/par-cstef/cache-91983bfecf21472d244f6bf26116bc20ee4433d3/a.exe Show quoted text

> ... > 0x0000000000000001 (NEEDED) Shared library: [libperl.so.5.8] > ... > $ ls /home/users/cstef/tmp/par-cstef/cache- > 91983bfecf21472d244f6bf26116bc20ee4433d3/libperl* > /home/users/cstef/tmp/par-cstef/cache- > 91983bfecf21472d244f6bf26116bc20ee4433d3/libperl.so.5.8.8

That explains it: - the a.exe in the cache directory (which is different from the a.exe you actually invoked) is actually a special purpose Perl interpreter (it's actually the same file for any packed executable) - since your perl (on the machine where you built PAR::Packer) is linked against a shared libperl, the special interpreter is also linked against this libperl - in this case we also pack said shared libperl into the executable; if the packed executable is run, the packed libperl is extracted into the cache area The problem is the name of the libperl file in the cache: when linking, ld burned "libperl.so.5.8" into a.exe, so that's the name the shared library loader is looking for when trying to execute a.exe (the one in the cache area). It is run with the cache area prepended to LD_LIBRARY_PATH, so that's where the loader is looking first. But the file there is called "libperl.so.5.8.8", hence it won't get considered. Next on the loader's search path is the built-in /usr/lib where a file called libperl.so.5.8 is actually found - but it's from perl 5.8.9 (and there exist apparently slight incompatibilities between 5.8.8 and 5.8.9 on the C level). So we've to find out what caused the discrepancy libperl.so.5.8 vs libperl.so.5.8.8 (and perhaps improve PAR::Packer's build process to avoid that). Can you please post the output of perl -V (on the machine where you built PAR::Packer)? IIRC I've seen similar before - are you running CentOS? Note: Your workaround confirms that the problem is indeed caused by the "wrong" libperl.so, but it won't work in general as the cache area is specific to the user and also specific to the packed executable (the long checksum in the cache area name is approximately the SHA1 of the executable itself).

Thu Nov 18 04:46:20 2010 cstef [...] parallel.ru - Correspondence added

From:

cstef [...] parallel.ru

On Thu Nov 18 04:38:06 2010, RSCHUPP wrote: Show quoted text

> So we've to find out what caused the discrepancy libperl.so.5.8 > vs libperl.so.5.8.8 (and perhaps improve PAR::Packer's build process > to avoid that). Can you please post the output of > > perl -V

[coctic@t60-2 ~]$ perl -V Summary of my perl5 (revision 5 version 8 subversion 8 patch 33449) configuration: Platform: osname=linux, osvers=2.6.18-ovz-smp-alt24, archname=x86_64-linux-thread-multi uname='linux apiary.rio.altlinux.org 2.6.18-ovz-smp-alt24 #1 smp tue may 6 19:11:08 msd 2008 x86_64 gnulinux ' config_args='-de -rs -Darchname=x86_64-linux -Dd_dosuid -Ud_csh -Dlibswanted=dl m c crypt db ndbm gdbm -Duseshrplib -Dlibperl=libperl.so.5.8 -Dcc=gcc -Doptimize=-pipe -Wall -O2 -D_GNU_SOURCE -Dcccdlflags=-fPIC -DPIC -Dccdlflags=-rdynamic -Wl,-O1 -Dlddlflags=-shared -Wl,-O1 -Dldflags=-Wl,-O1 -Dprefix=/usr -Dprivlib=/usr/lib/perl5 -Darchlib=/usr/lib/perl5/x86_64-linux -Dvendorprefix=/usr -Dvendorlib=/usr/lib/perl5/vendor_perl -Dvendorarch=/usr/lib/perl5/vendor_perl/x86_64-linux -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dvendorman1dir=/usr/share/man/man1 -Dvendorman3dir=/usr/share/man/man3 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/lib/perl5/site_perl/5.8.8 -Dsitearch=/usr/local/lib/perl5/site_perl/5.8.8/x86_64-linux -Dsiteman1dir=/usr/local/man/man1 -Dsite_man3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3pm -Dcf_by=ALT Linux Team -Dcf_email=Alexey Tourbin <at@altlinux.ru> -Dmyhostname=localhost -Dperladmin=root@localhost -Dmyuname=Linux 2.6.18-ovz-smp-alt24 x86_64 -Dnewmyuname=Linux 2.6.18-ovz-smp-alt24 x86_64 -Dinc_version_list=5.8.7/x86_64-linux 5.8.6/x86_64-linux 5.8.5/x86_64-linux 5.8.4/x86_64-linux 5.8.3/x86_64-linux 5.8.2/x86_64-linux 5.8.1/x86_64-linux 5.8.0/x86_64-linux 5.8.7 5.8.6 5.8.5 5.8.4 5.8.3 5.8.2 5.8.1 5.8.0 5.6.1 5.6.0 -Dpager=/usr/bin/less -isR -Di_shadow -Di_syslog -Dusethreads -Duseithreads -Duselargefiles -Di_db -Di_gdbm -Di_ndbm -Di_sdbm -Ui_odbm' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm', optimize='-pipe -Wall -O2 -D_GNU_SOURCE', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -pipe -I/usr/local/include -I/usr/include/gdbm' ccversion='', gccversion='4.1.2 20070626 (ALT Linux, build 4.1.2-alt2)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='gcc', ldflags ='-Wl,-O1 -L/usr/local/lib64 -L/usr/local/lib' libpth=/usr/local/lib64 /usr/local/lib /lib64 /usr/lib64 /lib /usr/lib libs=-ldl -lm -lpthread -lc -lcrypt -ldb -lgdbm perllibs=-ldl -lm -lpthread -lc -lcrypt libc=/lib64/libc-2.5.1.so, so=so, useshrplib=true, libperl=libperl.so.5.8 gnulibc_version='2.5.1' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-O1' cccdlflags='-fPIC -DPIC', lddlflags='-shared -Wl,-O1 -L/usr/local/lib64 -L/usr/local/lib' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP THREADS_HAVE_PIDS USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_REENTRANT_API Locally applied patches: MAINT33449 ALT Linux patches Built under linux Compiled at Sep 7 2008 16:38:34 %ENV: PERL5LIB="/home/coctic/perl/lib/perl5/site_perl/:/home/coctic/perl/lib/perl5/site_perl/5.8.8/:/home/coctic/modules/lib/perl5/:/home/coctic/modules/lib/perl5/x86_64-linux-thread-multi/" @INC: /home/coctic/perl/lib/perl5/site_perl//5.8.8 /home/coctic/perl/lib/perl5/site_perl/ /home/coctic/perl/lib/perl5/site_perl/5.8.8/ /home/coctic/modules/lib/perl5//x86_64-linux-thread-multi /home/coctic/modules/lib/perl5/ /home/coctic/modules/lib/perl5/x86_64-linux-thread-multi/ /etc/perl5 /usr/lib/perl5/x86_64-linux /usr/lib/perl5 /usr/local/lib/perl5/site_perl/5.8.8/x86_64-linux /usr/local/lib/perl5/site_perl/5.8.8 /usr/local/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/x86_64-linux /usr/lib/perl5/vendor_perl /usr/lib/perl5/vendor_perl . Show quoted text

> (on the machine where you built PAR::Packer)? > IIRC I've seen similar before - are you running CentOS?

It is not CentOS, it is AltLinux, special version for high performance clusters. I am not sure if it has connections with CentOS. Show quoted text

> Note: Your workaround confirms that the problem is indeed caused by > the "wrong" libperl.so, but it won't work in general as the > cache area is specific to the user and also specific to > the packed executable (the long checksum in the cache area name > is approximately the SHA1 of the executable itself).

I understand it, and I would definitely prefer the bug to be fixed somehow, and I am ready to do my best for it.

Fri Nov 19 02:44:18 2010 RSCHUPP [...] cpan.org - Correspondence added

On 2010-11-18 04:46:20, Coctic wrote: Show quoted text

> somehow, and I am ready to do my best for it.

All right! Please run the attached script (on the machine where you built PAR::Packer) and post its output. It's actually the code from myldr/Makefile.PL to find the shared libperl with some modifications. Cheers, Roderich

Subject:

find-libperl.pl

#!/usr/bin/perl -w # Copyright 2002-2009 by Audrey Tang. # Copyright (c) 2002 Mattia Barbon. # This package is free software; you can redistribute it and/or # modify it under the same terms as Perl itself. use strict; use warnings; use Config; use File::Spec; use ExtUtils::Embed; use DynaLoader; use File::Basename; xsinit(undef); # used for searching libperls. sub find_file { my $file = shift; my @paths = ( $Config{bin}, File::Spec->catdir($Config{'archlibexp'}, 'CORE'), split(/\Q$Config{path_sep}\E/, $ENV{$Config{ldlibpthname}} || ''), split(/ /, $Config{libpth}), ); my $libperl; if ($libperl = DynaLoader::dl_findfile("-lperl")) { if (-l $libperl) { my $realpath = readlink($libperl); if (!File::Spec->file_name_is_absolute($realpath)) { $realpath = File::Spec->rel2abs( File::Spec->catfile( dirname($libperl) => $realpath ) ); } $libperl = $realpath; } return $libperl if -e $libperl; } foreach my $path (@paths) { $libperl = File::Spec->catfile($path, $file); return $libperl if -e $libperl; # for MinGW $libperl = File::Spec->catfile($path, $1) if $file =~ /^lib(.+)/; return $libperl if -e $libperl; # for Cygwin $libperl = File::Spec->catfile($path, $file.$Config{_a}); return $libperl if -e $libperl; } } my $pldflags = ldopts(); my $dynperl = $Config{useshrplib} && ($Config{useshrplib} ne 'false'); $dynperl = 1 if $pldflags =~ /\B-lperl\b/; # Gentoo lies to us! my $libperl; if ($dynperl) { my $file = $Config{libperl}; my $so = $Config{so} || 'so'; $file = "libperl.$so" if $file eq 'libper'; # workaround Red Hat bug $file =~ s/\.(?!\d)[^.]*$/.$Config{so}/; $file =~ s/^lib// if $^O eq "MSWin32"; $libperl = find_file($file); if (not -e $libperl) { $file =~ s/\.(?!\d)[^.]*$/.a/; $libperl = find_file($file); } } else { print "no libperl DSO in use\n"; exit 0; } if (-e $libperl) { print "libperl is \"$libperl\"\n"; my $name = basename($libperl); if ($^O =~ /linux/) { my ($soname) = qx(objdump -ax $libperl) =~ /^\s*SONAME\s+(\S+)/m; if ($? == 0 && defined $soname) { print "soname is \"$soname\"\n"; $name = $soname; } } print "link name is \"$name\"\n"; } else { print "libperl appears to be \"$libperl\", but it doesn't exist\n"; } # local variables: # mode: cperl # end:

Fri Nov 19 02:53:30 2010 cstef [...] parallel.ru - Correspondence added

From:

cstef [...] parallel.ru

On Fri Nov 19 02:44:18 2010, RSCHUPP wrote: Show quoted text

> All right! Please run the attached script > (on the machine where you built PAR::Packer) > and post its output. It's actually the code

[coctic@t60-2 tmp]$ ./find-libperl.pl libperl is "/usr/lib64/libperl.so.5.8.8" soname is "libperl.so.5.8" link name is "libperl.so.5.8" [coctic@t60-2 tmp]$

Sun Nov 21 09:13:01 2010 RSCHUPP [...] cpan.org - Correspondence added

On 2010-11-19 02:53:30, Coctic wrote: Show quoted text

> [coctic@t60-2 tmp]$ ./find-libperl.pl > libperl is "/usr/lib64/libperl.so.5.8.8" > soname is "libperl.so.5.8" > link name is "libperl.so.5.8"

That shows the problem (basename of libperl is different from the "link name") and also how to fix it (at least on Linux using the GNU binutils toolchain). Can you please: - drop myldr/{Makefile.PL,file2c.pl} from the attached tarball into a clean PAR::Packer source - perl Makefile.PL ; make ; make test ; make install - re-pack your test script and try it out Cheers, Roderich

Subject:

63083.tar.gz

Download 63083.tar.gz
application/x-gzip 4.9k

Message body not shown because it is not plain text.

Sun Nov 21 10:42:02 2010 cstef [...] parallel.ru - Correspondence added

From:

cstef [...] parallel.ru

Yes, test binary works after transfer to another machine as well as my other binaries, thank a lot. May I hope that the fix will be in the next release of PAR::Packer?

Sun Nov 21 12:14:23 2010 RSCHUPP [...] cpan.org - Taken

Sun Nov 21 12:14:43 2010 RSCHUPP [...] cpan.org - Correspondence added

On 2010-11-21 10:42:02, Coctic wrote: Show quoted text

> Yes, test binary works after transfer to another machine as well as my > other binaries, thank a lot.

Thanks for testing, I've just committed the fix. Show quoted text

> May I hope that the fix will be in the next release of PAR::Packer?

I just uploaded PAR::Packer 1.008 to PAUSE (contains this and other assorted fixes). Cheers, Roderich

Sun Nov 21 12:14:44 2010 RSCHUPP [...] cpan.org - Status changed from 'open' to 'resolved'