Re: gettext-0.14.2 fails on parallel build

[Bruno Haible]

Alexandre Duret-Lutz wrote:
> Make starts only one command per target, and wait for this
> target to be completed before building any other dependent
> targets.
> 
> The only way to have the same target built several times is to
> explicitly start separate make processes yourself.  (That was
> what was happening.)
>
> When make encounters the following fragment
>
> all: one.elc two.elc three.elc
>
> one.elc two.elc three.elc: elc-stamp
>         ...
>
> it first has to build the dependency: elc-stamp.  Only one
> process does this.  Then, and only after elc-stamp has been
> built, make can run the three "..." commands in parallel.  It
> won't start the "..." commands their dependencies (here elc-stamp)
> are built.
>
> In Greg's output this is clearly apparent.  The elc-stamp is run
> a first time alone.  After it has finished, three `make
> elc-stamp' are explicitly started by the three parallel instance
> of the "..." rule.

You are right with your analysis, answering the question "*why* are there
three processes in parallel, executing the elc-stamp simultaneously".

And now that they are executing in parallel, what happens is:

        Process 1 creates elc-temp.
        Process 2 re-creates elc-temp.
        Then process 1 and process 2 perform the "..." task.
        Process 1 moves elc-temp to elc-stemp.
        Process 2 attempts to do so as well, but elc-temp was already gone.

Your fix replaced a line inside the $(am__ELCFILES) actions, so that it does
nothing if $(EMACS) = no. So it will fix the problem for Greg - until
he installs an Emacs, does "make", "rm -f *.elc" and "make" again. Then the
same problem will occur again.

Please look again at the Makefile.in. It contains

am__ELFILES = po-compat.el po-mode.el start-po.el
am__ELCFILES = $(am__ELFILES:.el=.elc)

$(am__ELCFILES): elc-stamp
        @if test ! -f $@; then \
          rm -f elc-stamp; \
          $(MAKE) $(AM_MAKEFLAGS) elc-stamp; \
        else : ; fi

elc-stamp: $(LISP)
        @echo 'WARNING: Warnings can be ignored. :-)'
        @rm -f elc-temp && touch elc-temp
        if test "$(EMACS)" != no; then \
          set x; \
          list='$(LISP)'; for p in $$list; do \
            if test -f "$$p"; then d=; else d="$(srcdir)/"; fi; \
            set x "$$@" "$$d$$p"; shift; \
          done; \
          shift; \
          EMACS="$(EMACS)" $(SHELL) $(elisp_comp) "$$@" || exit 1; \
        else : ; fi
        @mv -f elc-temp $@

The $(am__ELCFILES) is, as you say, responsible for spawning three
'make' processes trying to create elc-stamp.

What is the purpose of this rule? It could mean: "If you just remade
elc-stamp, but it didn't produce all required *.elc files, then remake
elc-stamp again." But that is not useful since executing the same commands
a second time will not produce a better result.

So it must mean: "If the user has removed *.elc but elc-stamp is still
there, then remake elc-stamp." This scenario can still occur, and will
still lead to multiple parallel processes [except if only one .elc file
was missing], and will still fail the same way.

> The automake's aux scripts use mkdir as atomic test-and-set.

Ah, right. Thanks. So here is an updated fix proposal:

elc-stamp: $(LISP)
    trap 'rm -rf elc-temp elc-stamp' 1 2 3 15; \
    if mkdir elc-temp 2>/dev/null; then \
      # Comment: This code is being executed by the first process.
      if test "$(EMACS)" != no; then \
        [compile the *.lisp files] \
      else : ; fi; \
      touch $@; \
      rmdir elc-temp; \
    else \
      # Comment: This code is being executed by the follower processes.
      # Comment: Wait until the first process is done.
      while test -d elc-temp; do sleep 1; done; \
      # Comment: Succeed if and only if the first process succeeded.
      test -f $@; exit $$?; \
    fi

Bruno