Bug #63939 for PAR-Packer: The behaviour of "pp --link ..." is subtly different between OSX and Linux.

Wed Dec 15 04:00:06 2010 PHILKIME [...] cpan.org - Ticket created

It seems that the behaviour of --link is subtly different between OSX and Linux. Many libraries are symbolic links and are referenced through the link names. Consider a library referenced as X which is a symlink to Y X -> Y On linux, pp dereferences X and packs Y with the name X, which is nice. However on OSX, it seems that pp dereferences X and packs Y but with the name Y. This breaks components which reference the library via name X. I can work around this by copying Y to name X in, say, /tmp and then packing /tmp/X and this works but it would be nice to make OSX pp have the same behaviour as linux pp.

Wed Dec 15 10:57:08 2010 RSCHUPP [...] cpan.org - Correspondence added

On 2010-12-15 04:00:06, PHILKIME wrote: Show quoted text

> However on OSX, it seems that pp dereferences X and packs Y but with the > name Y.

I suppose you checked the cache area of your packed executable (typically /tmp/par-USR/SHA1-CHECKSUM) and found Y there, right? Show quoted text

> This breaks components which reference the library via name X.

Looks like the heuristic in methods _find_shlib and _chase_lib of PAR::Packer is wrong for OSX. Sorry, I know next to nothing about OSX - does it use ELF for objects and shared libraries? If so, what is the SONAME of X? Can you run the following script with a single argument: the value of pp's --link option where the problem shows? use PAR::Packer; my ($link) = @ARGV; my $pp = PAR::Packer->new; print $pp->_find_shlib($link, "foobar"); If it complains about "Environment variable LD_LIBRARY_PATH does not exist", just set LD_LIBRARY_PATH to the built-in library search path (e.g. /lib:/usr/lib on Linux, Solaris). Cheers, Roderich

Wed Dec 15 10:57:09 2010 The RT System itself - Status changed from 'new' to 'open'

Thu Feb 09 03:05:04 2012 RSCHUPP [...] cpan.org - Subject changed from (no value) to 'The behaviour of "pp --link ..." is subtly different between OSX and Linux.'

Fri Oct 03 09:08:13 2014 lamprecht [...] cpan.org - Correspondence added

The naming schema for shared libs on darwin is libname.x.x.x.dylib The attached patch changes _find_shlib and _chase_lib to use the extension .dylib instead of of $Config{dlext} which has the value '.bundle' (there are different types of dynamically linked libs). It also changes the regexes used to derive the correct name 'libname.x.dylib' from the forms -l libname.dylib ; -l libname.x.x.x.dylib; /path/to/libname.dylib etc. Files that are named .*\.bundle (these do not follow the version numbering schema of .dylib files) can still be referenced by their full name. I'm a pp user and not an expert concerning MacOSX. So anyone who could review would be welcome. Cheers, Christoph Show quoted text

> > Looks like the heuristic in methods _find_shlib and _chase_lib > of PAR::Packer is wrong for OSX. Sorry, I know next to nothing > about OSX - does it use ELF for objects and shared libraries? > If so, what is the SONAME of X? > > Can you run the following script with a single argument: > the value of pp's --link option where the problem shows? > > use PAR::Packer; > my ($link) = @ARGV; > my $pp = PAR::Packer->new; > print $pp->_find_shlib($link, "foobar"); > > If it complains about "Environment variable LD_LIBRARY_PATH > does not exist", just set LD_LIBRARY_PATH to the built-in library > search path (e.g. /lib:/usr/lib on Linux, Solaris).

Subject:

PAcker.pm.patch

diff --git a/lib/PAR/Packer.pm b/lib/PAR/Packer.pm index 2017f6c..b85bc2c 100644 --- a/lib/PAR/Packer.pm +++ b/lib/PAR/Packer.pm @@ -1448,6 +1448,8 @@ sub _check_par { sub _chase_lib { my ($self, $file) = @_; + if ($^O eq q/darwin/){return _chase_lib_darwin(@_)} + while ($Config::Config{d_symlink} and -l $file) { if ($file =~ /^(.*?\.\Q$Config{dlext}\E\.\d+)\..*/) { return $1 if -e $1; @@ -1470,6 +1472,35 @@ sub _chase_lib { return $file; } +sub _chase_lib_darwin{ + my ($self, $file) = @_; + + while (-l $file) { + if ($file =~ /^(.*?\.\d+)(\.\d+)*\.dylib$/) { + my $name = $1 . q/.dylib/; + return $name if -e $name; + } + + return $file if $file =~ /\D\.\d+\.dylib$/; + + my $dir = File::Basename::dirname($file); + $file = readlink($file); + + unless (File::Spec->file_name_is_absolute($file)) { + $file = File::Spec->rel2abs($file, $dir); + } + } + + if ($file =~ /^(.*?\.\d+)(\.\d+)*\.dylib$/) { + my $name = $1 . q/.dylib/; + return $name if -e $name; + } + + return $file; + +} + + sub _find_shlib { my ($self, $file, $script_name) = @_; @@ -1497,14 +1528,14 @@ sub _find_shlib { . " does not exist.\n"; return; } - + my $dlext = ($^O eq 'darwin') ? 'dylib' : ($Config{dlext} // ''); $file = File::Basename::basename($file); for my $dir (File::Basename::dirname($0), split(/\Q$Config{path_sep}\E/, $libpthname)) { my $abs = File::Spec->catfile($dir, $file); return $self->_chase_lib($abs) if -e $abs; - $abs = File::Spec->catfile($dir, "$file.$Config{dlext}"); + $abs = File::Spec->catfile($dir, "$file.$dlext"); return $self->_chase_lib($abs) if -e $abs; }

Subject:

Shell.txt

utas-mbp:~ chris$ perl -MPAR::Packer -e 'print $PAR::Packer::VERSION' 1.022 utas-mbp:pptest chris$ cat > test.pl print "PID: $$\n"; print "PAR_TEMP: ",$ENV{PAR_TEMP}, "\n"; use DBD::Pg; system qq/lsof -p $$ | grep libpq /; while( 1 ){sleep 5}; utas-mbp:pptest chris$ pp -o test -l /opt/local/lib/postgresql93/libpq.5.6.dylib test.pl utas-mbp:pptest chris$ ./test PID: 416 PAR_TEMP: /var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T//par-6368726973/cache-bd93cb56c5d7783d814fe8bbb37720f2c98d740a test 416 chris txt REG 1,2 147476 1235950 /opt/local/lib/postgresql93/libpq.5.6.dylib # the packed executable still uses the systems lib # the correct file is extracted in our cache-dir but the name is wrong # so it won't get loaded (should be libpq.5.dylib): utas-mbp:~ chris$ ls /var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T//par-6368726973/cache-bd93cb56c5d7783d814fe8bbb37720f2c98d740a | grep libpq libpq.5.6.dylib # With Patched Packer.pm: utas-mbp:pptest chris$ pp -o test -l libpq test.pl Can't find libpq. Environment variable DYLD_LIBRARY_PATH does not exist. utas-mbp:pptest chris$ export DYLD_LIBRARY_PATH=/opt/local/lib/postgresql93/ utas-mbp:pptest chris$ pp -o test -l libpq test.pl utas-mbp:pptest chris$ ./test PID: 371 PAR_TEMP: /var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T//par-6368726973/cache-dc89f0112a238b2b842d8d4f4e95baf2bd31ebb9 test 371 chris txt REG 1,1 147476 1502680 /private/var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T/par-6368726973/cache-dc89f0112a238b2b842d8d4f4e95baf2bd31ebb9/libpq.5.dylib ^C utas-mbp:pptest chris$ pp -o test -lpq test.pl Unknown option: lpq utas-mbp:pptest chris$ utas-mbp:pptest chris$ pp -o test -l pq test.pl utas-mbp:pptest chris$ ./test PID: 384 PAR_TEMP: /var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T//par-6368726973/cache-ebfb6618b3d0d3e9f0267d9f9ad8feda90a4b3a8 test 384 chris txt REG 1,1 147476 1503982 /private/var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T/par-6368726973/cache-ebfb6618b3d0d3e9f0267d9f9ad8feda90a4b3a8/libpq.5.dylib ^C utas-mbp:pptest chris$ pp -o test -l libpq.5.6.dylib test.pl utas-mbp:pptest chris$ ./test PID: 393 PAR_TEMP: /var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T//par-6368726973/cache-c419a78fda3a47f5d1133b65163b3eaa3539c823 test 393 chris txt REG 1,1 147476 1505279 /private/var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T/par-6368726973/cache-c419a78fda3a47f5d1133b65163b3eaa3539c823/libpq.5.dylib ^C utas-mbp:pptest chris$ pp -o test -l libpq.dylib test.pl utas-mbp:pptest chris$ ./test PID: 402 PAR_TEMP: /var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T//par-6368726973/cache-dbce9b62835facfe961713d5dbe13c03274d40d4 test 402 chris txt REG 1,1 147476 1506575 /private/var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T/par-6368726973/cache-dbce9b62835facfe961713d5dbe13c03274d40d4/libpq.5.dylib ^C utas-mbp:pptest chris$ pp -o test -l /opt/local/lib/postgresql93/libpq.5.6.dylib test.pl utas-mbp:pptest chris$ ./test PID: 411 PAR_TEMP: /var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T//par-6368726973/cache-15c3bd67b417a2e74de810a7fc03a50d7bb99b69 test 411 chris txt REG 1,1 147476 1507875 /private/var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T/par-6368726973/cache-15c3bd67b417a2e74de810a7fc03a50d7bb99b69/libpq.5.dylib ^C utas-mbp:pptest chris$ pp -o test -l /opt/local/lib/postgresql93/libpq.dylib test.pl utas-mbp:pptest chris$ ./test PID: 420 PAR_TEMP: /var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T//par-6368726973/cache-53b72d8635b1ad3d625a281db44177d3b2571168 test 420 chris txt REG 1,1 147476 1509171 /private/var/folders/52/p5_rmm7x2nx7t773fwkdyhj00000gp/T/par-6368726973/cache-53b72d8635b1ad3d625a281db44177d3b2571168/libpq.5.dylib ^C utas-mbp:pptest chris$

Sun Mar 27 10:39:25 2016 PHILKIME [...] cpan.org - Correspondence added

Strange I didn't see this reply so many years ago ... here is the output, I still use PAR::Packer and still have the workaround in place: Show quoted text

> ls -ld /opt/local/lib/libz.1.dylib

lrwxr-xr-x 1 root admin 16 2 Oct 22:33 /opt/local/lib/libz.1.dylib -> libz.1.2.8.dylib Show quoted text

> /tmp/thing.pl /opt/local/lib/libz.1.dylib

/opt/local/lib/libz.1.dylib Does this mean this is fixed now?

Tue Mar 29 04:48:14 2016 RSCHUPP [...] cpan.org - Correspondence added

On 2016-03-27 10:39:25, PHILKIME wrote: Show quoted text

> Does this mean this is fixed now?

Maybe... I applied Christoph Lamprecht's for PAR::Packer 1.024. Feel free to reopen this bug if the problem persists. Cheers, Roderich

Tue Mar 29 04:48:21 2016 RSCHUPP [...] cpan.org - Status changed from 'open' to 'resolved'