tracing from aclocal (Was: Re: [Bug-gnulib] proposed gettext.m4 patch for inttypes.h (merge from coreutils))

[Alexandre Duret-Lutz]

>>> "Bruno" == Bruno Haible <address@hidden> writes:

[...]

 Bruno> Can someone please fix 'aclocal' so that, after
 Bruno> determining which macros are potentially needed, it runs
 Bruno> autoconf with tracing enabled, to determine which macros
 Bruno> are really needed?

 Bruno> The alternative is to recommend that people don't define
 Bruno> autoconf macros with arguments, and that only autoconf
 Bruno> macros that take no arguments are created and
 Bruno> documented. I think this "solution" would be inferior...

Right now I think it's ok to have arguments provided you don't
call macros _conditionally_ inside your macros.

Anyway, the following patch (against CVS HEAD) updates aclocal
to build aclocal.m4 in two steps as you suggested.

  1a) scan for potential macros
  1b) build aclocal.m4 with all potential macros
  2a) trace for actual macros
  2b) build aclocal.m4 with all actual macros

I have two problem with this change.  I think both of them
should be addressed in Autoconf, but maybe I'll have other
ideas.

  * This breaks tests/aclocal7.test.  This test make sure
    we do not overwrite aclocal.m4 needlessly (i.e., when
    the content is the same and no dependency is newer).

    Obviously this is impossible to do if aclocal.m4 has
    to be written twice.  

    I could try to reset the timestamp of aclocal.m4 afterwards
    in case the file really shouldn't have changed, but that
    sound clumsy.  Especially there will still exist a period
    where aclocal.m4 is fresh, and that is likely to trouble
    parallel makes.

    What I'd like is that step 1b) builds a temporary file,
    say aclocal.tmp, and step 2a) tells autoconf to use
    this temporary file instead of aclocal.m4.  

    Maybe autom4te already allows this?  Akim?
  
  * If aclocal calls autoconf --trace, then the following
    call to autoconf does not print any diagnostics.  So

       aclocal && autoconf -Wall

    has exactly the same effect as
      
       aclocal && autoconf

    only the standard diagnostic are issued, and this is 
    done while aclocal run.  This is the reason for
    the change in tests/obsolete.test in the patch below.

    Not that this second issue is quite independent from
    this patch.  I think people running 
    `automake && autoconf -Wall' instead of `autoconf -Wall && automake'
    get into the same trouble (although both uses are equally valid
    from the Makefile dependencies point of view).


BTW you owe me a fuse.  The circuit breaker tripped during `make
check' (thanks God for ext3), and I still haven't plugged the
fridge back.

 
2003-08-12  Alexandre Duret-Lutz  <address@hidden>

        * aclocal.in (write_aclocal): Take a list of used macros in
        argument and construct aclocal.m4 here.
        (trace_used_macros): New function.
        (add_file): Do not update $output.
        ($output): Delete.
        (MAIN): Write aclocal.m4 for all seen macros.  Then trace for
        these macros, and rewrite aclocal.m4 only for these traced macros.
        This should shorten aclocal.m4 by stripping out unused macros.
        * tests/aclibobj.test: Make sure configure.in exists by the time
        aclocal runs.
        * tests/obsolete.test: Erase autom4te.cache.
        * tests/aclocal8.test: New file.
        * tests/Makefile.am (TESTS): Add aclocal8.test.
        Suggested by Bruno Haible.

Index: NEWS
===================================================================
RCS file: /cvs/automake/automake/NEWS,v
retrieving revision 1.221
diff -u -r1.221 NEWS
--- NEWS        27 Jul 2003 12:58:38 -0000      1.221
+++ NEWS        12 Aug 2003 21:56:39 -0000
@@ -71,6 +71,11 @@
     Autom4te's cache isn't needlessly invalidated.  This behavior can
     be switched off with the new `--force' option.
 
+  - aclocal now uses Autoconf's --trace to detect macros which are actually
+    used and will no longer include unused macros simply because they
+    where mentioned.  This were often the case for macros called
+    conditionally.
+
   - New option no-dist-gzip.
 
   - install-sh now understands --version and --help.
Index: aclocal.in
===================================================================
RCS file: /cvs/automake/automake/aclocal.in,v
retrieving revision 1.84
diff -u -r1.84 aclocal.in
--- aclocal.in  7 Aug 2003 00:32:29 -0000       1.84
+++ aclocal.in  12 Aug 2003 21:56:40 -0000
@@ -55,9 +55,6 @@
 # Exit status.
 $exit_status = 0;
 
-# Text to output.
-$output = '';
-
 # Output file name.
 $output_file = 'aclocal.m4';
 
@@ -100,9 +97,11 @@
 &scan_m4_files (@dirlist);
 &scan_configure;
 if (! $exit_status)
-{
-    &write_aclocal;
-}
+  {
+    &write_aclocal (keys %macro_seen);
+    my %macro_traced = &trace_used_macros;
+    &write_aclocal (keys %macro_traced);
+  }
 &check_acinclude;
 
 exit $exit_status;
@@ -375,22 +374,6 @@
     return if ($file_seen{$file});
     $file_seen{$file} = 1;
 
-    my $mtime = mtime $file;
-    $greatest_mtime = $mtime if $greatest_mtime < $mtime;
-
-    # If the file to add looks like path outside the project,
-    # copy it to the output.
-    # The regex catches filenames starting with things like
-    #   / \ c:\ ../ ./../ etc.
-    if ($file =~ m,^(?:(?:\w:)?[\\/]|(?:\.[\\/]+)*\.\.[\\/]),)
-      {
-       $output .= $file_contents{$file} . "\n";
-      }
-    else
-      {
-       # Otherwise, simply include the file.
-       $output .= "m4_include([$file])\n";
-      }
     my (@rlist);
     foreach (split ("\n", $file_contents{$file}))
     {
@@ -456,22 +439,64 @@
     return $contents;
 }
 
+sub trace_used_macros ()
+{
+  my $traces = ($ENV{AUTOCONF} || 'autoconf') . " ";
+  $traces .= join (' ', map { "--trace='$_:\$n'" } (keys %macro_seen));
+  my $tracefh = new Automake::XFile ("$traces $configure_ac |");
+
+  my %traced = ();
+
+  while ($_ = $tracefh->getline)
+    {
+      chomp;
+      $traced{$_} = 1 if $macro_seen{$_};
+    }
+  return %traced;
+}
+
 ################################################################
 
 # Write output.
-sub write_aclocal ()
+sub write_aclocal (@)
 {
-    # Nothing to output?!
-    # FIXME: Shouldn't we diagnose this?
-    return if ! length ($output);
-
-# We used to print `# $output_file generated automatically etc.'  But
-# this creates spurious differences when using autoreconf.  Autoreconf
-# creates aclocal.m4t and then rename it to aclocal.m4, but the
-# rebuild rules generated by Automake create aclocal.m4 directly --
-# this would gives two ways to get the same file, with a different
-# name in the header.
-    $output = "# generated automatically by aclocal $VERSION -*- Autoconf -*-
+  my @macros = @_;
+  my $output = '';
+
+  my %files = map { $map{$_} => 1 } @macros;
+  $files{'acinclude.m4'} = 1 if -f 'acinclude.m4';
+
+  for $file (sort keys %files)
+    {
+      my $mtime = mtime $file;
+      $greatest_mtime = $mtime if $greatest_mtime < $mtime;
+
+      # If the file to add looks like path outside the project,
+      # copy it to the output.
+      # The regex catches filenames starting with things like
+      #   / \ c:\ ../ ./../ etc.
+      if ($file =~ m,^(?:(?:\w:)?[\\/]|(?:\.[\\/]+)*\.\.[\\/]),)
+       {
+         $output .= $file_contents{$file} . "\n";
+       }
+      else
+       {
+         # Otherwise, simply include the file.
+         $output .= "m4_include([$file])\n";
+       }
+    }
+
+  # Nothing to output?!
+  # FIXME: Shouldn't we diagnose this?
+  return if ! length ($output);
+
+  # We used to print `# $output_file generated automatically etc.'  But
+  # this creates spurious differences when using autoreconf.  Autoreconf
+  # creates aclocal.m4t and then rename it to aclocal.m4, but the
+  # rebuild rules generated by Automake create aclocal.m4 directly --
+  # this would gives two ways to get the same file, with a different
+  # name in the header.
+  $output = "# generated automatically by aclocal $VERSION -*- Autoconf -*-
 
 # Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003
 # Free Software Foundation, Inc.
@@ -486,29 +511,29 @@
 
 $output";
 
-    # We try not to update $output_file unless necessary, because
-    # doing so invalidate Autom4te's cache and therefore slows down
-    # tools called after aclocal.
-    #
-    # We need to overwrite $output_file in the following situations.
-    #   * The --force option is in use.
-    #   * One of the dependencies is younger.
-    #     (Not updating $output_file in this situation would cause
-    #     make to call aclocal in loop.)
-    #   * The contents of the current file are different from what
-    #     we have computed.
-    if (!$force_output
-       && $greatest_mtime < mtime ($output_file)
-       && $output eq contents ($output_file))
-      {
-       print STDERR "aclocal: $output_file unchanged\n" if $verbose;
-       return;
-      }
+  # We try not to update $output_file unless necessary, because
+  # doing so invalidate Autom4te's cache and therefore slows down
+  # tools called after aclocal.
+  #
+  # We need to overwrite $output_file in the following situations.
+  #   * The --force option is in use.
+  #   * One of the dependencies is younger.
+  #     (Not updating $output_file in this situation would cause
+  #     make to call aclocal in loop.)
+  #   * The contents of the current file are different from what
+  #     we have computed.
+  if (!$force_output
+      && $greatest_mtime < mtime ($output_file)
+      && $output eq contents ($output_file))
+    {
+      print STDERR "aclocal: $output_file unchanged\n" if $verbose;
+      return;
+    }
 
-    print STDERR "aclocal: writing $output_file\n" if $verbose;
+  print STDERR "aclocal: writing $output_file\n" if $verbose;
 
-    my $out = new Automake::XFile "> $output_file";
-    print $out $output;
+  my $out = new Automake::XFile "> $output_file";
+  print $out $output;
 }
 
 ### Setup "GNU" style for perl-mode and cperl-mode.
Index: tests/Makefile.am
===================================================================
RCS file: /cvs/automake/automake/tests/Makefile.am,v
retrieving revision 1.509
diff -u -r1.509 Makefile.am
--- tests/Makefile.am   9 Aug 2003 22:34:30 -0000       1.509
+++ tests/Makefile.am   12 Aug 2003 21:56:40 -0000
@@ -10,6 +10,7 @@
 aclocal5.test \
 aclocal6.test \
 aclocal7.test \
+aclocal8.test \
 acoutnoq.test \
 acoutpt.test \
 acoutpt2.test \
Index: tests/aclibobj.test
===================================================================
RCS file: /cvs/automake/automake/tests/aclibobj.test,v
retrieving revision 1.3
diff -u -r1.3 aclibobj.test
--- tests/aclibobj.test 8 Sep 2002 13:07:54 -0000       1.3
+++ tests/aclibobj.test 12 Aug 2003 21:56:40 -0000
@@ -1,5 +1,5 @@
 #! /bin/sh
-# Copyright (C) 2001, 2002  Free Software Foundation, Inc.
+# Copyright (C) 2001, 2002, 2003  Free Software Foundation, Inc.
 #
 # This file is part of GNU Automake.
 #
@@ -22,12 +22,10 @@
 
 . ./defs || exit 1
 
-cat > X << 'END'
-AC_INIT
-AM_INIT_AUTOMAKE(nonesuch, nonesuch)
+cat >> configure.in << 'END'
 AC_PROG_CC
 AC_PROG_RANLIB
-AC_OUTPUT(Makefile)
+AC_OUTPUT
 END
 
 cat > Makefile.am << 'END'
@@ -43,7 +41,7 @@
 
 set -e
 
-cp X configure.in
+cp configure.in X
 echo 'AC_LIBSOURCE(maude.c)' >> configure.in
 $AUTOMAKE
 
Index: tests/aclocal8.test
===================================================================
RCS file: tests/aclocal8.test
diff -N tests/aclocal8.test
--- /dev/null   1 Jan 1970 00:00:00 -0000
+++ tests/aclocal8.test 12 Aug 2003 21:56:40 -0000
@@ -0,0 +1,44 @@
+#! /bin/sh
+# Copyright (C) 2003  Free Software Foundation, Inc.
+#
+# This file is part of GNU Automake.
+#
+# GNU Automake is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# GNU Automake is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Automake; see the file COPYING.  If not, write to
+# the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+# Boston, MA 02111-1307, USA.
+
+# Make sure aclocal require unused macros.
+
+. ./defs || exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+SOME_DEFS
+END
+
+mkdir m4
+cat >m4/somedefs.m4 <<EOF
+AC_DEFUN([SOME_DEFS], [
+  m4_if([a], [a], [MACRO1], [MACRO2])
+])
+EOF
+
+echo 'AC_DEFUN([MACRO1],)' >m4/macro1.m4
+echo 'AC_DEFUN([MACRO2],)' >m4/macro2.m4
+
+$ACLOCAL -I m4
+grep macro1.m4 aclocal.m4
+grep macro2.m4 aclocal.m4 && exit 1
+:
Index: tests/obsolete.test
===================================================================
RCS file: /cvs/automake/automake/tests/obsolete.test,v
retrieving revision 1.9
diff -u -r1.9 obsolete.test
--- tests/obsolete.test 8 Sep 2002 13:07:55 -0000       1.9
+++ tests/obsolete.test 12 Aug 2003 21:56:40 -0000
@@ -1,5 +1,5 @@
 #! /bin/sh
-# Copyright (C) 1996, 2001, 2002  Free Software Foundation, Inc.
+# Copyright (C) 1996, 2001, 2002, 2003  Free Software Foundation, Inc.
 #
 # This file is part of GNU Automake.
 #
@@ -36,6 +36,9 @@
 test `cat configure.in | wc -l` -gt 1 || exit 1
 
 $ACLOCAL || exit 1
+
+# Erase Autom4te's cache so that Autoconf prints its diagnostics.
+rm -Rf autom4te*.cache
 
 # Expect Autoconf to complain about each of the macros in obs.
 $AUTOCONF -Wobsolete >stderr 2>&1

-- 
Alexandre Duret-Lutz