Re: patch: Makefile.in in subdir not generated

[Ralf Wildenhues]

Hello Ilya,

Continuing the discussion started here:
<http://lists.gnu.org/archive/html/bug-automake/2008-08/msg00023.html>
Please do not start a new thread when replying, thank you.

* Ilya N. Golubev wrote on Wed, Aug 20, 2008 at 08:43:53PM CEST:
> Described what is fixing here in <Makefile.in in subdir not
> generated> posted to <address@hidden> on
> generated> (<address@hidden>).

OK, the situation is as follows: if a subdir/Makefile.in has been
removed, then the rebuild rules don't regenerate it correctly.

This is caused by the following: when subdir/Makefile.in is missing,
the rule for $(srcdir)/Makefile.in in subdir/Makefile has $? set to
all of $(am__configure_deps).  The current logic is that, if any
prerequisite is from $(am__configure_deps), then we just remake top
level target am--refresh.  This relies on the notion that, if
subdir/Makefile.in is out of date wrt. configure dependencies, then
toplevel Makefile.in should be as well, causing a complete rerun of
all autotools.

That logic is wrong in this case: for the toplevel Makefile, everything
is up to date.

> Patched 1.10.1 as follows.

>         * lib/am/configure.am (%MAKEFILE-IN%): Fixed `exit'ing before
>         invoking `$(AUTOMAKE)' if prerequisite other then
>         `Makefile.am' is newer.  Borrowed invocation of recursive
>         `make' and handling its exit codes from commands generated by
>         automake for other recursive targets.  Added elimination of
>         duplicate `$(AUTOMAKE)' invocation.

Thank you for your proposed patch.  It's a good starting point.
However, it causes duplicate automake reruns in the common case
where toplevel Makefile.am is updated.  Also, it lacks a test case.

Further, I'm not sure wheter -k should have an impact on the rebuilding
rules of prerequisites of `Makefile'.  Here's why it might be dangerous:
When some Makefile.in is out of date, and the rule rebuilding it were
destined to fail for some reason, and you run "make -k dist", then your
change will let the 'dist' target succeed, AFAICS.  But that would let
the wrong Makefile.in end up in the tarball.

I've pushed the patch below to master, and put you in THANKS (not shown,
neither any regenerated files).  The first test ensures that rebuilding
works for Makefile.in's in the common cases (updated Makefile.am, or
removed Makefile.in), and that automake is never called more than once
for each case.  The second case ensures that 'make' fails when any
rebuilding rule would fail.

I'm still pondering whether to put this in branch-1-10 as well.

Cheers, and thanks again,
Ralf

2008-08-22  Ralf Wildenhues  <address@hidden>

        Fix rebuilding of removed subdir/Makefile.in files.
        * lib/am/configure.am (%MAKEFILE-IN%) [?!TOPDIR_P?]: If
        subdir/Makefile.in was removed, am--refresh would not update
        it.  Fix up for it by running the per-directory rebuild rule.
        * tests/remake6.test, tests/remake7.test: New tests.
        * tests/Makefile.am: Adjust.
        * NEWS, THANKS: Update.
        Report and initial patch by Ilya N. Golubev.

diff --git a/NEWS b/NEWS
index 181ba05..bd1f369 100644
--- a/NEWS
+++ b/NEWS
@@ -55,6 +55,9 @@ New in 1.10a:
     stub rules.  This allows `make' to trigger a rerun of `aclocal'
     also if some previously needed macro file has been removed.
 
+  - Rebuild rules now also work for a removed `subdir/Makefile.in' in
+    an otherwise up to date tree.
+
   - The `color-tests' option causes colored test result output on terminals.
 
   - New prefix `notrans_' for manpages which should not be transformed
diff --git a/lib/am/configure.am b/lib/am/configure.am
index 879f837..4623f41 100644
--- a/lib/am/configure.am
+++ b/lib/am/configure.am
@@ -1,5 +1,5 @@
 ## automake - create Makefile.in from Makefile.am
-## Copyright (C) 2001, 2002, 2003, 2004, 2006, 2007  Free Software
+## Copyright (C) 2001, 2002, 2003, 2004, 2006, 2007, 2008  Free Software
 ## Foundation, Inc.
 
 ## This program is free software; you can redistribute it and/or modify
@@ -41,8 +41,13 @@ endif %?TOPDIR_P%
            *$$dep*) \
 ?TOPDIR_P?           echo ' cd $(srcdir) && $(AUTOMAKE) --%STRICTNESS% 
%USE-DEPS%'; \
 ?TOPDIR_P?           cd $(srcdir) && $(AUTOMAKE) --%STRICTNESS% %USE-DEPS% \
-?!TOPDIR_P?          cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh 
\
-               && exit 0; \
+?TOPDIR_P?             && exit 0; \
+?!TOPDIR_P?          ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) 
am--refresh ) \
+## If on the other hand, subdir/Makefile.in has been removed, then toplevel
+## am--refresh will not be aware of any need to run.  We still invoke it
+## due to $? listing all prerequisites.  Fix up for it by running the rebuild
+## rule for this file only, below.
+?!TOPDIR_P?            && { if test -f $@; then exit 0; else break; fi; }; \
              exit 1;; \
          esac; \
        done; \
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 45624af..1e19657 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -496,6 +496,8 @@ remake2.test \
 remake3.test \
 remake4.test \
 remake5.test \
+remake6.test \
+remake7.test \
 regex.test \
 req.test \
 reqd.test \
diff --git a/tests/remake6.test b/tests/remake6.test
new file mode 100755
index 0000000..b94dda3
--- /dev/null
+++ b/tests/remake6.test
@@ -0,0 +1,64 @@
+#! /bin/sh
+# Copyright (C) 2008  Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Make sure remaking rules work when subdir Makefile.in has been removed.
+
+. ./defs || exit 1
+set -e
+
+cat > configure.in << 'END'
+AC_INIT
+AM_INIT_AUTOMAKE(nonesuch, nonesuch)
+AC_OUTPUT(Makefile sub/Makefile)
+END
+
+cat > Makefile.am <<'END'
+SUBDIRS = sub
+END
+mkdir sub
+: > sub/Makefile.am
+
+$ACLOCAL
+$AUTOMAKE
+$AUTOCONF
+./configure
+$MAKE
+
+# Now, we are set up.  Ensure that, for either missing Makefile.in,
+# or updated Makefile.am, rebuild rules are run, and run exactly once
+# only.
+
+rm -f Makefile.in
+$MAKE >stdout
+cat stdout
+test `grep -c " --run " stdout` -eq 1
+
+rm -f sub/Makefile.in
+$MAKE >stdout
+cat stdout
+test `grep -c " --run " stdout` -eq 1
+
+$sleep  # Let touched files appear newer.
+
+touch Makefile.am
+$MAKE >stdout
+cat stdout
+test `grep -c " --run " stdout` -eq 1
+
+touch sub/Makefile.am
+$MAKE >stdout
+cat stdout
+test `grep -c " --run " stdout` -eq 1
diff --git a/tests/remake7.test b/tests/remake7.test
new file mode 100755
index 0000000..7207285
--- /dev/null
+++ b/tests/remake7.test
@@ -0,0 +1,70 @@
+#! /bin/sh
+# Copyright (C) 2008  Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Make sure remaking rules fail when they should.
+
+. ./defs || exit 1
+set -e
+
+cat > configure.in << 'END'
+AC_INIT
+AM_INIT_AUTOMAKE(nonesuch, nonesuch)
+AC_OUTPUT(Makefile sub/Makefile)
+END
+
+cat > Makefile.am <<'END'
+SUBDIRS = sub
+END
+mkdir sub
+: > sub/Makefile.am
+
+$ACLOCAL
+$AUTOMAKE
+$AUTOCONF
+./configure
+$MAKE
+
+$sleep
+
+# Fail for broken input files.
+
+echo "if FOO" > sub/Makefile.am
+$MAKE && exit 1
+
+: >sub/Makefile.am
+$MAKE
+
+mv Makefile.am backup
+echo "if FOO" > Makefile.am
+$MAKE && exit 1
+
+# Fail for missing input files, with or without missing
+# Makefile.in files.
+
+cp backup Makefile.am
+mv sub/Makefile.am sub/backup
+$MAKE && exit 1
+
+rm -f sub/Makefile.in
+$MAKE && exit 1
+
+mv sub/backup sub/Makefile.am
+rm -f Makefile.am
+$MAKE && exit 1
+
+rm -f Makefile.in
+$MAKE && exit 1
+: