Skip Menu |

This queue is for tickets about the ExtUtils-MakeMaker CPAN distribution.

Report information
The Basics
Id: 129290
Status: open
Priority: 0/
Queue: ExtUtils-MakeMaker
People
Owner: Nobody in particular
Requestors: nospam-abuse [...] ilyaz.org
Cc:
AdminCc:
Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)

History Show all quoted text
  Tue Apr 23 03:30:18 2019 nospam-abuse [...] ilyaz.org - Ticket created
Subject: Wrong order of dependencies generated in presence of MYEXTLIB in a subdirectory
Date: Tue, 23 Apr 2019 00:30:05 -0700
To: bug-ExtUtils-MakeMaker [...] rt.cpan.org
From: Ilya Zakharevich <nospam-abuse [...] ilyaz.org>
Hello, I’m fighting with this issue (on and off) for decades now. Anyway, the generated Makefile contains something like this (following the docs in the POD): all :: pure_all $(NOECHO) $(NOOP) pure_all :: config pm_to_blib subdirs linkext $(NOECHO) $(NOOP) subdirs :: $(MYEXTLIB) $(NOECHO) $(NOOP) subdirs :: $(NOECHO) cd sdbm && $(MAKE) $(USEMAKEFILE) $(FIRST_MAKEFILE) all $(PASTHRU) $(MYEXTLIB): cd sdbm && $(MAKE) $(PASTHRU) As far as I understand, as a result, cd sdbm && etc is executed twice: once as one of the ::-rules for subdirs, once as a dependence of $(MYEXTLIB). With parallel make's (as a lot of machines on Perl-Testers network do) these child $(MAKE) may run concurrently, leading to disasters. ⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜⁜ Proposal: change the second `subdirs´ rule to subdirs :: subdirs-makeall subdirs-makeall :: $(NOECHO) cd sdbm && $(MAKE) $(USEMAKEFILE) $(FIRST_MAKEFILE) all $(PASTHRU) Then one could just specify `subdirs-makeall´ as a dependency for $(MYEXTLIB). (The example of disaster is in http://www.cpantesters.org/cpan/report/c935d292-632a-11e9-bf31-80c71e9d5857 although there it is due to TRIPLE dependence on a rule with `cd FOO && …´.) Yours, Ilya P.S. I see no way to fix this in forward-compatible way; the Makefile.PL⸣s must be version-dependent to use `depend on subdirs-makeall´ trick. P.P.S. With several subdirectories, one can have multiples: subdir-makeall1 subdir-makeall2 subdir-makeall3 … to speed up the parallel make…
  Tue Apr 23 07:46:27 2019 The RT System itself - Status changed from 'new' to 'open'
  Tue Apr 23 11:30:03 2019 ETJ [...] cpan.org - Correspondence added
I tried something like a double-level dependency, adding a target pure_nolink, which pure_all would depend on. This triggered a gmake bug, since fixed in the latest, but there will be gmakes in the wild without the fix. Therefore, a double-level dependency is not a viable solution. I am also very skeptical, based on experience, about advocating a backwards-incompatible solution. There are distros out there, including DBD::Pg (I think, else it was mysql), that can't handle parallel testing. It wouldn't surprise me at all to learn there are some that can't handle parallel makes. However, the example you cite, of sdbm, is obviously part of Perl itself, which gets rigorously exercised using parallel make whenever Perl is built. I would therefore be stunned if the default EUMM approach breaks. Certainly the deps you have shown are wrong, in that there are two separate rules that cd into "sdbm" (very misleading example, since sdbm doesn't have this problem). Are you SURE the problem doesn't lie with Math-Pari's expression of dependencies? In particular, this: $(AUTOGEN_HEADERS) $(MYEXTLIB): libPARI/Makefile FORCE cd libPARI && $(MAKE) $(PASTHRU) looks wrong. I would assume you actually want something like this: $(AUTOGEN_HEADERS) $(MYEXTLIB): libPARI/Makefile $(NOECHO) $(NOOP) libPARI/Makefile: cd libPARI && $(MAKE) $(PASTHRU) Getting parallel right is all about expressing the dependencies right. It's not easy!