Bug #68108 for ZeroMQ: ZMQ Detection on OSX fails

Wed May 11 05:33:47 2011 johan [...] stromnet.se - Ticket created

Subject:	ZMQ Detection on OSX fails
Date:	Wed, 11 May 2011 11:33:24 +0200
To:	bug-ZeroMQ [...] rt.cpan.org
From:	Johan Ström <johan [...] stromnet.se>

Hi, I'm trying to install ZeroMQ-0.12 manually on OS X (perl v5.8.8), with zeromq 2.1.6 installed locally, but I've got some problems with the autodetection of zmq. Steps to reproduce: $ export DEST=$HOME/dev/inst $ mkdir -p $DEST # First install zeromq locally $ tar zxf zeromq-2.1.1.tar.gz $ cd zeromq-2.1. $ ./configure --prefix=$DEST ... $ make ... $ make install ... # Now try to build ZeroMQ perl $ tar zxf ZeroMQ-0.12.tar.gz $ cd ZeroMQ-0.12 $ env ZMQ_HOME=$DEST perl Makefile.PL Can't link/include 'zmq.h', 'zmq' $ env ZMQ_HOME=$DEST PKG_CONFIG_PATH=$DEST/lib/pkgconfig perl Makefile.PL Can't link/include 'zmq.h', 'zmq' I tried a bit of re-arranging in Makefile.pl, moving checklibs from line 27 to after the detect_zmq.pl-stuff, which yielded this: $ env PKG_CONFIG_PATH=$DEST/lib/pkgconfig perl Makefile.PL Probing environment variables: Probing libzmq via pkg-config ... + found libzmq 2.1.6 + Detected ZMQ_INCLUDES from pkg-config... + Detected ZMQ_H from pkg-config... + Detected ZMQ_LIBS from pkg-config... Detected the following ZMQ settings: + ZMQ_HOME = (null) + ZMQ_H = /Users/johan/dev/inst/include/zmq.h + ZMQ_INCLUDES = /Users/johan/dev/inst/include + ZMQ_LIBS = -L/Users/johan/dev/inst/lib -lzmq + ZMQ_TRACE = (null) Can't link/include 'zmq.h', 'zmq' With a bit more specific args I got it working, but this shouldn't be necessary: $ env PKG_CONFIG_PATH=$DEST/lib/pkgconfig perl Makefile.PL INC="-I $DEST/include -I./" LIBS="-L$DEST/lib" PREFIX="$DEST" Probing environment variables: Probing libzmq via pkg-config ... + found libzmq 2.1.6 + Detected ZMQ_INCLUDES from pkg-config... + Detected ZMQ_H from pkg-config... + Detected ZMQ_LIBS from pkg-config... Detected the following ZMQ settings: + ZMQ_HOME = (null) + ZMQ_H = /Users/johan/dev/inst/include/zmq.h + ZMQ_INCLUDES = /Users/johan/dev/inst/include + ZMQ_LIBS = -L/Users/johan/dev/inst/lib -lzmq + ZMQ_TRACE = (null) Writing ppport.h + Using zmq.h from /Users/johan/dev/inst/include/zmq.h Checking if your kit is complete... Looks good Warning: prerequisite Devel::PPPort 3.19 not found. We have 3.0601. Warning: prerequisite ExtUtils::MakeMaker 6.42 not found. We have 6.30. Warning: prerequisite Task::Weaken 0 not found. Warning: prerequisite Test::Exception 0.29 not found. Warning: prerequisite Test::More 0.98 not found. We have 0.62. Warning: prerequisite Test::Requires 0 not found. Warning: prerequisite Test::TCP 0 not found. Writing Makefile for ZeroMQ And make works out fine aswell $ make cp lib/ZeroMQ/Context.pm blib/lib/ZeroMQ/Context.pm cp lib/ZeroMQ/Poller.pm blib/lib/ZeroMQ/Poller.pm cp lib/ZeroMQ/Constants.pm blib/lib/ZeroMQ/Constants.pm cp lib/ZeroMQ.pm blib/lib/ZeroMQ.pm cp lib/ZeroMQ/Socket.pm blib/lib/ZeroMQ/Socket.pm cp lib/ZeroMQ/Message.pm blib/lib/ZeroMQ/Message.pm cp lib/ZeroMQ/Raw.pm blib/lib/ZeroMQ/Raw.pm /opt/local/bin/perl "-Iinc" /opt/local/lib/perl5/5.8.8/ExtUtils/ xsubpp -typemap /opt/local/lib/perl5/5.8.8/ExtUtils/typemap xs/ perl_zeromq.xs > xs/perl_zeromq.xsc && mv xs/perl_zeromq.xsc xs/ perl_zeromq.c cc -c -I/Users/johan/dev/inst/include -I./ -I/opt/local/include -fno- common -DPERL_DARWIN -no-cpp-precomp -fno-strict-aliasing -pipe - Wdeclaration-after-statement -I/opt/local/include -Wall -Wextra - Wdeclaration-after-statement -O3 -DVERSION=\"0.12\" -DXS_VERSION= \"0.12\" -o xs/perl_zeromq.o "-I/opt/local/lib/perl5/5.8.8/ darwin-2level/CORE" -DUSE_PPPORT xs/perl_zeromq.c perl_zeromq.xs: In function 'XS_ZeroMQ__Raw_zmq_poll': perl_zeromq.xs:588: warning: comparison between signed and unsigned perl_zeromq.xs:589: warning: comparison between signed and unsigned perl_zeromq.xs:608: warning: passing argument 4 of 'Perl_hv_fetch' makes integer from pointer without a cast perl_zeromq.xs:619: warning: passing argument 4 of 'Perl_hv_fetch' makes integer from pointer without a cast perl_zeromq.xs:628: warning: passing argument 4 of 'Perl_hv_fetch' makes integer from pointer without a cast perl_zeromq.xs:636: warning: passing argument 4 of 'Perl_hv_fetch' makes integer from pointer without a cast perl_zeromq.c: At top level: xs/mg-xs.inc:56: warning: 'PerlZMQ_Raw_PollItem_vtbl' defined but not used Running Mkbootstrap for ZeroMQ () chmod 644 ZeroMQ.bs rm -f blib/arch/auto/ZeroMQ/ZeroMQ.bundle env MACOSX_DEPLOYMENT_TARGET=10.3 cc -L/opt/local/lib -bundle - undefined dynamic_lookup xs/perl_zeromq.o -o blib/arch/auto/ZeroMQ/ ZeroMQ.bundle \ \ chmod 755 blib/arch/auto/ZeroMQ/ZeroMQ.bundle cp ZeroMQ.bs blib/arch/auto/ZeroMQ/ZeroMQ.bs chmod 644 blib/arch/auto/ZeroMQ/ZeroMQ.bs Manifying blib/man3/ZeroMQ::Poller.3 Manifying blib/man3/ZeroMQ::Context.3 Manifying blib/man3/ZeroMQ::Constants.3 Manifying blib/man3/ZeroMQ.3 Manifying blib/man3/ZeroMQ::Socket.3 Manifying blib/man3/ZeroMQ::Raw.3 Manifying blib/man3/ZeroMQ::Message.3 So, to conclude, the detection doesn't really seem to utilize the ZMQ_* env properly.. The above works fine with the clean Makefile.PL aswell (ie no moving around of checklibs), but I supposed that is due to my explicit INC/LIBS flags. Am I missing something, or can the detection stuff be further improved? For the record, I'm working (rather passively though) on a POE integration, which can be found at https://github.com/stromnet/poe-zeromq . Best Regards Johan

Wed May 11 19:23:53 2011 dmaki [...] cpan.org - Correspondence added

Can you please see if the latest master works for you? https://github.com/lestrrat/ZeroMQ-Perl The relevant change is: https://github.com/lestrrat/ZeroMQ- Perl/commit/383ba71b7dafbe44b418c6e901e221da54f41828

Wed May 11 19:23:54 2011 The RT System itself - Status changed from 'new' to 'open'

Thu May 12 02:26:50 2011 johan [...] stromnet.se - Correspondence added

Subject:	Re: [rt.cpan.org #68108] ZMQ Detection on OSX fails
Date:	Thu, 12 May 2011 08:26:35 +0200
To:	bug-ZeroMQ [...] rt.cpan.org
From:	Johan Ström <johan [...] stromnet.se>

That seems to have done the trick! I cheated a bit and used the inc dir from the old distr instead of installing Module::Install and all other "development-mode" extensions, not sure if that makes any difference but now it works! To avoid using dev mode/install dev modules: $ mv .git .. $ cp -r ../ZeroMQ-0.12/inc . $ env ZMQ_HOME=$DEST perl Makefile.PL Probing environment variables: + Detected ZMQ_INCLUDES from ZMQ_HOME... + Detected ZMQ_H from ZMQ_HOME... + Detected ZMQ_LIBS from ZMQ_HOME... Probing libzmq via pkg-config ... Package libzmq was not found in the pkg-config search path. Perhaps you should add the directory containing `libzmq.pc' to the PKG_CONFIG_PATH environment variable No package 'libzmq' found - No libzmq found... Probing zeromq2 via pkg-config ... Package zeromq2 was not found in the pkg-config search path. Perhaps you should add the directory containing `zeromq2.pc' to the PKG_CONFIG_PATH environment variable No package 'zeromq2' found - No zeromq2 found... Detected the following ZMQ settings: + ZMQ_HOME = /Users/johan/dev/inst + ZMQ_H = /Users/johan/dev/inst/include/zmq.h + ZMQ_INCLUDES = /Users/johan/dev/inst/include + ZMQ_LIBS = -L/Users/johan/dev/inst/lib + ZMQ_TRACE = (null) Writing ppport.h + Using zmq.h from /Users/johan/dev/inst/include/zmq.h Checking if your kit is complete... Warning: the following files are missing in your kit: eg/pushpull_client.pl eg/pushpull_server.pl META.yml Please inform the author. Warning: prerequisite Devel::PPPort 3.19 not found. We have 3.0601. Warning: prerequisite ExtUtils::MakeMaker 6.42 not found. We have 6.30. Warning: prerequisite Task::Weaken 0 not found. Warning: prerequisite Test::Exception 0.29 not found. Warning: prerequisite Test::More 0.98 not found. We have 0.62. Warning: prerequisite Test::Requires 0 not found. Warning: prerequisite Test::TCP 0 not found. Writing Makefile for ZeroMQ $ make ... Make completes perfectly fine! Using PKG_CONFIG_PATH (only) works fine as well. I guess that fixed it, thank you! On May 12, 2011, at 01:23 , Daisuke Maki via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=68108 > > > Can you please see if the latest master works for you? https://github.com/lestrrat/ZeroMQ-Perl > > > The relevant change is: https://github.com/lestrrat/ZeroMQ- > Perl/commit/383ba71b7dafbe44b418c6e901e221da54f41828

Thu May 12 03:58:48 2011 dmaki [...] cpan.org - Correspondence added

On 2011-5月-12 木 02:26:50, johan@stromnet.se wrote: Show quoted text

> That seems to have done the trick!

Good! pushed and uploaded to CPAN.

Thu May 12 03:58:49 2011 dmaki [...] cpan.org - Status changed from 'open' to 'resolved'

Sun May 15 11:58:12 2011 johan [...] stromnet.se - Correspondence added

Subject:	Re: [rt.cpan.org #68108] ZMQ Detection on OSX fails
Date:	Sun, 15 May 2011 17:57:54 +0200
To:	bug-ZeroMQ [...] rt.cpan.org
From:	Johan Ström <johan [...] stromnet.se>

Hi again, seems some bugs got into the repo in https://github.com/lestrrat/ZeroMQ-Perl/commit/c6530482149817c6774b74ac783fb2e78ce32be7 and https://github.com/lestrrat/ZeroMQ-Perl/commit/31d8c22dd21bdcfd30b238bbe98604ad12c098e1 - my $version = qx/pkg-config --modversion $pkg/; + my $version = qx/$pkg_config --modversion $pkg/; and similar in the other patch. That actually tries to call the PKG_CONFIG_PATH specified (which is one or more directories), instead of executing the pkg-config command: Probing environment variables: Probing libzmq via /Users/johan/dev/inst/lib/pkgconfig ... - No libzmq found... Probing zeromq2 via /Users/johan/dev/inst/lib/pkgconfig ... - No zeromq2 found... The following patch (against current master) seems to fix it: diff --git a/tools/detect_zmq.pl b/tools/detect_zmq.pl index 3498ba0..7344557 100644 --- a/tools/detect_zmq.pl +++ b/tools/detect_zmq.pl @@ -68,7 +68,7 @@ sub probe_pkgconfig { my $pkg_config = $ENV{ PKG_CONFIG_PATH } || 'pkg-config'; foreach my $pkg ( qw(libzmq zeromq2) ) { print "Probing $pkg via $pkg_config ...\n"; - my $version = qx/$pkg_config --modversion $pkg/; + my $version = qx/pkg-config --modversion $pkg/; chomp $version; if (! $version) { print " - No $pkg found...\n"; @@ -77,7 +77,7 @@ sub probe_pkgconfig { print " + found $pkg $version\n"; if (! $ENV{ZMQ_INCLUDES}) { - if (my $cflags = qx/$pkg_config --cflags-only-I $pkg/) { + if (my $cflags = qx/pkg-config --cflags-only-I $pkg/) { chomp $cflags; print " + Detected ZMQ_INCLUDES from $pkg_config... \n"; my @paths = map { s/^-I//; $_ } split /\s+/, $cflags; @@ -96,7 +96,7 @@ sub probe_pkgconfig { } if (! $ENV{ZMQ_LIBS}) { - if (my $libs = qx/$pkg_config --libs $pkg/) { + if (my $libs = qx/pkg-config --libs $pkg/) { chomp $libs; print " + Detected ZMQ_LIBS from $pkg_config...\n"; $ENV{ZMQ_LIBS} = $libs; @@ -106,4 +106,4 @@ sub probe_pkgconfig { last; } The env variable is inherited properly, and the libraries are detected fine. At least on OS X.. On FreeBSD however, I get the following problem: $ env PKG_CONFIG_PATH=$DEST/lib/pkgconfig perl Makefile.PL PREFIX="$DEST" Probing environment variables: Probing libzmq via /home/johan/dev/inst/lib/pkgconfig ... + found libzmq 2.1.7 + Detected ZMQ_INCLUDES from /home/johan/dev/inst/lib/pkgconfig... + Detected ZMQ_H from /home/johan/dev/inst/lib/pkgconfig... + Detected ZMQ_LIBS from /home/johan/dev/inst/lib/pkgconfig... Detected the following ZMQ settings: + ZMQ_HOME = (null) + ZMQ_H = /home/johan/dev/inst/include/zmq.h + ZMQ_INCLUDES = /home/johan/dev/inst/include + ZMQ_LIBS = -L/home/johan/dev/inst/lib -lzmq + ZMQ_TRACE = (null) sys_cmd -> /usr/bin/cc -DAPPLLIB_EXP="/usr/local/lib/perl5/5.10.1/ BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing - pipe -fstack-protector -I/usr/local/include -Wl,-E -fstack-protector - L/usr/local/lib assertlibXzdSrdHW.c -o assertlibS2F3So8b -lzmq -I/home/ johan/dev/inst/include -L/home/johan/dev/inst/lib at inc/Devel/ CheckLib.pm line 309. /libexec/ld-elf.so.1: Shared object "libzmq.so.1" not found, required by "assertlibS2F3So8b" Very wierd, the -L/home/johan/dev/inst/lib flag is there.. and libzmq.so.1 is there: $ ls -al $DEST/lib/ total 12179 drwxr-xr-x 3 johan johan 7 May 15 14:32 . drwxr-xr-x 5 johan johan 5 May 15 14:32 .. -rw-r--r-- 1 johan johan 9251320 May 15 14:32 libzmq.a -rwxr-xr-x 1 johan johan 950 May 15 14:32 libzmq.la lrwxr-xr-x 1 johan johan 11 May 15 14:32 libzmq.so -> libzmq.so.1 -rwxr-xr-x 1 johan johan 3126612 May 15 14:32 libzmq.so.1 drwxr-xr-x 2 johan johan 3 May 15 14:32 pkgconfig After hacking in the "-Wl,--verbose" param to @sys_cmd in inc/Devel/ CheckLib.pm, I got the following from ld: ... attempt to open /var/tmp//cctZv99Z.o succeeded /var/tmp//cctZv99Z.o attempt to open /usr/local/lib/libzmq.so failed attempt to open /usr/local/lib/libzmq.a failed attempt to open /home/johan/dev/inst/lib/libzmq.so succeeded -lzmq (/home/johan/dev/inst/lib/libzmq.so) ...(nothing more relevant)... libm.so.5 needed by /home/johan/dev/inst/lib/libzmq.so found libm.so.5 at /lib/libm.so.5 /libexec/ld-elf.so.1: Shared object "libzmq.so.1" not found, required by "assertlibbvaiHD_K" Very wierd, seems like its finding the "wrong" library or something, but I don't know enough LD-magic to know what's going on here. If i skip the whole assertlib stuff it builds fine though, so the paths/library is working fine. Johan On May 12, 2011, at 09:58 , Daisuke Maki via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=68108 > > > On 2011-5月-12 木 02:26:50, johan@stromnet.se wrote:

>> That seems to have done the trick!

> > > Good! pushed and uploaded to CPAN.

Sun May 15 11:58:14 2011 The RT System itself - Status changed from 'resolved' to 'open'

Sun May 15 18:14:52 2011 dmaki [...] cpan.org - Correspondence added

that should've already been fixed. did you git pull? https://github.com/lestrrat/ZeroMQ- Perl/commit/31d8c22dd21bdcfd30b238bbe98604ad12c098e1 --d

Sun May 15 18:14:53 2011 dmaki [...] cpan.org - Status changed from 'open' to 'resolved'

Mon May 16 03:45:25 2011 johan [...] stromnet.se - Correspondence added

Subject:	Re: [rt.cpan.org #68108] ZMQ Detection on OSX fails
Date:	Mon, 16 May 2011 09:45:11 +0200
To:	bug-ZeroMQ [...] rt.cpan.org
From:	Johan Ström <johan [...] stromnet.se>

Yep, latest master as of yesterday when I tested. The commit you linked to is the same one I linked to, which is the one which breaks the pkg-config execuction. The $pkg_config var is sourced from the PKG_CONFIG_PATH env var, and $pkg_config is then executed. my $pkg_config = $ENV{ PKG_CONFIG_PATH } || 'pkg-config'; foreach my $pkg ( qw(libzmq zeromq2) ) { print "Probing $pkg via $pkg_config ...\n"; my $version = qx/$pkg_config --modversion $pkg/; chomp $version; if (! $version) { print " - No $pkg found...\n"; next; } From the man-page of pkg-config: .... By default, pkg-config looks in the directory prefix/lib/pkgconfig for these files; it will also look in the colon-separated (on Windows, semicolon-separated) list of directories specified by the PKG_CONFIG_PATH environment variable. Using PKG_CONFIG_PATH as the executable is thus wrong. Johan On May 16, 2011, at 00:14 , Daisuke Maki via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=68108 > > > that should've already been fixed. did you git pull? > https://github.com/lestrrat/ZeroMQ- > Perl/commit/31d8c22dd21bdcfd30b238bbe98604ad12c098e1 > > --d

Message body is not shown because it is too large.

Mon May 16 03:45:26 2011 The RT System itself - Status changed from 'resolved' to 'open'

Mon May 16 04:05:51 2011 dmaki [...] cpan.org - Correspondence added

oh oops? I guess I wasn't looking. sorry, I'm in a middle of a business trip, so I'll take a closer look later. Thanks for your patience.

Mon May 16 04:07:35 2011 johan [...] stromnet.se - Correspondence added

Subject:	Re: [rt.cpan.org #68108] ZMQ Detection on OSX fails
Date:	Mon, 16 May 2011 10:07:25 +0200
To:	bug-ZeroMQ [...] rt.cpan.org
From:	Johan Ström <johan [...] stromnet.se>

Understandable! Just glad I can contribute some :) On May 16, 2011, at 10:05 AM, Daisuke Maki via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=68108 > > > oh oops? I guess I wasn't looking. > sorry, I'm in a middle of a business trip, so I'll take a closer look later. > Thanks for your patience.

Sun May 22 19:01:26 2011 dmaki [...] cpan.org - Correspondence added

Okay, so I should source $pkg_config from some other env var? The only thing I'm trying to do with the $pkg_config = $ENV{ ... } || "pkg-config" line is to allow the user to specify an alternate pkg-config executable when running Makefile.PL does changing the env var name to, say, PKGCONFIG_CMD or something work?

Mon May 23 03:28:27 2011 johan [...] stromnet.se - Correspondence added

Subject:	Re: [rt.cpan.org #68108] ZMQ Detection on OSX fails
Date:	Mon, 23 May 2011 09:28:15 +0200
To:	bug-ZeroMQ [...] rt.cpan.org
From:	Johan Ström <johan [...] stromnet.se>

Hi, I'm not a pkg-config expert, but when do you need to specify an alternate binary? As far as I've understood, you have one pkg-config binary but you might have multiple locations with .pc files, and these locations is what you specify with PKG_CONFIG_PATH. If there is a need to use an alternate binary, I'd guess that something like that would work, but I've never seen that. Either way, PKG_CONFIG_PATH should however never be used for that. On May 23, 2011, at 1:01 AM, Daisuke Maki via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=68108 > > > Okay, so I should source $pkg_config from some other env var? > The only thing I'm trying to do with the > > $pkg_config = $ENV{ ... } || "pkg-config" > > line is to allow the user to specify an alternate pkg-config executable when running > Makefile.PL > > does changing the env var name to, say, PKGCONFIG_CMD or something work?

Mon May 23 03:44:15 2011 dmaki [...] cpan.org - Correspondence added

Show quoted text

> I'm not a pkg-config expert, but when do you need to specify an > alternate binary? As far as I've understood, you have one pkg- > config binary but you might have multiple locations with .pc files, > and these locations is what you specify with PKG_CONFIG_PATH.

I've never really used pkg-config other than for this purposes, so I don't know :) But from a defensive programming POV, I try to always make such things configurable. That's all. Does this work for you? 220718977538c77cf6dbd91bda1f2c174c87b497

Mon May 23 03:50:04 2011 johan [...] stromnet.se - Correspondence added

Subject:	Re: [rt.cpan.org #68108] ZMQ Detection on OSX fails
Date:	Mon, 23 May 2011 09:49:53 +0200
To:	bug-ZeroMQ [...] rt.cpan.org
From:	Johan Ström <johan [...] stromnet.se>

Sure, that should probably be working fine! On May 23, 2011, at 9:44 AM, Daisuke Maki via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=68108 > > >

>> I'm not a pkg-config expert, but when do you need to specify an >> alternate binary? As far as I've understood, you have one pkg- >> config binary but you might have multiple locations with .pc files, >> and these locations is what you specify with PKG_CONFIG_PATH.

> > I've never really used pkg-config other than for this purposes, so I don't know :) > > But from a defensive programming POV, I try to always make such things configurable. > That's all. > > Does this work for you? 220718977538c77cf6dbd91bda1f2c174c87b497

Mon May 23 04:20:46 2011 dmaki [...] cpan.org - Correspondence added

On 2011-5月-23 月 03:50:04, johan@stromnet.se wrote: Show quoted text

> Sure, that should probably be working fine!

Right on, new version coming right up... --d