RFC: 59-autom4te-cfg.patch

[akim]

I now consider Autotest is really usable: its users no longer need to
deal with hard coded paths etc.  I am not willing to open autom4te.cfg
to users yet, but I think we should do it sooner or later.  For
instance, Alexandre Lutz once told me it'd be a good thing to specify,
on a per project basis, additional trace preselections.  We can bet
that Automake too will need to trace more that what I listed below.
And it would be a pity to have to wait two or more runs to have the
trace cache reach its equilibrium.

So what is needed is a means to *extend* a language with additional
flags.  Something like $top_srcdir/autom4te.cfg seems fine.  Does it?

Similarly, ~/.autom4te.cfg seems quite logical.  OTOH, this might be
the beginning of a new sort of troubles...

There is one dirty thing that crept in here: the CLI options are
processed *after* the language options, which is of course the right
thing.  But there is one option which will bite us: --include.  I've
personally always considered it a bug in most tools that --include be
processed in the order it has been given.  Based on the high level
rule that the latest options always win, I think path walks should
always be performed in the reversed order of --includes.

In autom4te, I didn't do that because at first I wanted to stick to
GNU M4's own path walk.

Now that we use this scheme, we have two possibilities:

1. ask me to put some magic especially for the include path so that
   the user wins, but still reports the include path forwardly.

2. let's implement backward path walks.


You'll have to give me strong arguments in favor of 1 if you really
prefer this.  I'll soon implement 2.

Oh, BTW, this will certainly be yet a problem with make check: chances
are high that if you have some 2.52c installed, then make check will
chose the installed files instead of those in $build...  It works for
me, but hey, my installed 2.52c is fairly recent :)


Oh, also, distcheck is broken, for a simple reason, but yet I'm unsure
how to fix this properly.  It is related to the usual problem src vs
build.  Currently autom4te needs to find Struct.pm in src, and
autom4te.cfg in build, but has only a single relocation envvar:
AC_MACRODIR.  I'm not sure what to do, but there are many means to
work around this, so I'm not afraid.


Index: ChangeLog
from  Akim Demaille  <address@hidden>

        * lib/autom4te.in: New.
        * lib/Makefile.am (edit, autom4te.cfg): New.
        * bin/autom4te.in (BEGIN): Simplify.
        Rely on `AC_MACRODIR' instead of `autom4te_perllibdir'.
        (&load_configuration): New.  Use it.
        (&parse_args): Support --mode, --language, and --melt.
        * bin/autoconf.in: Simplify and adjust.
        * tests/Makefile.am (AUTOMAKE): Use --language.
        * tests/atlocal.in: Adjust.

Index: bin/Makefile.am
--- bin/Makefile.am Thu, 16 Aug 2001 11:39:27 +0200 akim
+++ bin/Makefile.am Mon, 27 Aug 2001 14:17:14 +0200 akim
@@ -46,10 +46,9 @@
        -e 's,@VERSION\@,$(VERSION),g' \
        -e 's,@PACKAGE_NAME\@,$(PACKAGE_NAME),g'

-## All the scripts below depend on configure.ac so that they are rebuilt
+## All the files below depend on configure.ac so that they are rebuilt
 ## when the Autoconf version changes. Unfortunately, suffix rules cannot
-## have additional dependencies, so we have to use explicit rules for
-## every script.
+## have additional dependencies, so we have to use explicit rules.

 autoconf: $(srcdir)/autoconf.in $(top_srcdir)/configure.ac
        rm -f autoconf autoconf.tmp
Index: bin/autoconf.in
--- bin/autoconf.in Thu, 16 Aug 2001 21:35:47 +0200 akim
+++ bin/autoconf.in Mon, 27 Aug 2001 16:56:55 +0200 akim
@@ -223,36 +223,29 @@ Tracing:
 # Unless specified, the output is stdout.
 test -z "$outfile" && outfile=-

-# Preselected macros: for autoheader and automake.
-for i in AC_CONFIG_HEADERS AH_OUTPUT AC_DEFINE_TRACE_LITERAL \
-         AC_SUBST AC_LIBSOURCE
-do
-  preselect="$preselect--preselect $i "
-done
-
+# Propagate -A.
+if test -n "$autoconf_dir"; then
+  export AC_MACRODIR=$autoconf_dir
+fi

 # Running autom4te.
 run_autom4te="$autom4te "\
 `$verbose "--verbose "`\
 `$debug && echo "--debug "`\
 `$force && echo "--force "`\
-"--normalize "\
-"--include $autoconf_dir --include $localdir "\
-"--warning syntax,$warnings "\
-"$preselect"\
-"autoconf/autoconf.m4"`$initialization || echo f`" "\
+"--language=autoconf "\
+"--include=$localdir "\
+`test -n "$warnings" && echo "--warning=$warnings "`\
+"--output=$outfile "\
+`$initialization && echo "--melt "`\
 `test -f "$autoconf_dir/acsite.m4" && echo "$autoconf_dir/acsite.m4"`" "\
 `test -f "$localdir/aclocal.m4" && echo "$localdir/aclocal.m4"`

 # Autom4te expansion.
 eval set dummy "$traces"
 shift
-$verbose "$me: running $run_autom4te "${1+"$@"}" $infile --output $outfile" >&2
-$run_autom4te ${1+"$@"} $infile --output $outfile ||
+$verbose "$me: running $run_autom4te "${1+"$@"}" $infile" >&2
+$run_autom4te ${1+"$@"} $infile ||
   { (exit 1); exit 1; }
-
-if test -z "$traces" && test "x$outfile" != x-; then
-  chmod +x $outfile
-fi

 (exit $status); exit $status
Index: bin/autom4te.in
--- bin/autom4te.in Sun, 19 Aug 2001 10:33:08 +0200 akim
+++ bin/autom4te.in Mon, 27 Aug 2001 17:15:22 +0200 akim
@@ -26,9 +26,8 @@

 BEGIN
 {
-  my $prefix = "@prefix@";
-  my $perllibdir = $ENV{'autom4te_perllibdir'} || "@datadir@";
-  unshift @INC, "$perllibdir";
+  my $datadir = $ENV{'AC_MACRODIR'} || '@datadir@';
+  unshift @INC, "$datadir";
 }

 ## --------- ##
@@ -235,6 +234,25 @@ sub load
 use IO::File;
 use strict;

+# Configuration file.
+my $datadir = $ENV{'AC_MACRODIR'} || '@datadir@';
+my $configuration_file = "$datadir/autom4te.cfg";
+
+# $LANGUAGE{$LANGUAGE} is the list of automatic options for $LANGUAGE.
+my %language;
+my $language;
+
+my $output = '-';
+
+# Should we normalize the output?
+my $normalize = 0;
+
+# Mode of the output file except for traces.
+my $mode = "0666";
+
+# If melt, don't use frozen files.
+my $melt = 0;
+
 # Names of the cache directory, cache directory index, trace cache
 # prefix, and output cache prefix.
 my $cache = "$me.cache";
@@ -253,11 +271,6 @@ sub load
 # FIXME: What about `sinclude'?
 my @preselect = ('include', 'm4_pattern_allow', 'm4_pattern_forbid');

-my $output = '-';
-
-# Should we normalize the output?
-my $normalize = 0;
-
 # Autom4te's default warnings, and the actual list of warnings.
 my @my_warning = ('syntax');
 my @warning;
@@ -394,6 +407,15 @@ Usage: $0 [OPTION] ... [FILES]
       --normalize          smash successive empty lines
   -f, --force              don\'t rely on cached values
   -W, --warnings=CATEGORY  report the warnings falling in CATEGORY
+  -l, --language=LANG      specify the set of M4 macros to use
+  -m, --mode=OCTAL         change the non trace output file mode (0666)
+  -M, --melt               don\'t use M4 frozen files
+
+Languages include:
+  \`Autoconf\'   create Autoconf configure scripts
+  \`Autotest\'   create Autotest test suites
+  \`M4sh\'       create M4sh shell scripts
+  \`M4sugar\'    create M4sugar output

 Warning categories include:
   \`cross\'         cross compilation issues
@@ -437,6 +459,46 @@ sub print_version
 }


+# load_configuration ()
+# ---------------------
+# Load the configuration file.
+sub load_configuration ()
+{
+  use Text::ParseWords;
+
+  my $cfg = new IO::File ($configuration_file)
+    or die "$me: cannot read $configuration_file: $!\n";
+  my $lang;
+  while ($_ = $cfg->getline)
+    {
+      chomp;
+      # Comments.
+      next
+       if /^\s*(\#.*)?$/;
+
+      my @words = shellwords ($_);
+      my $type = shift @words;
+      if ($type eq 'begin-language:')
+       {
+         $lang = lc $words[0];
+       }
+      elsif ($type eq 'end-language:')
+       {
+         die "$me: $configuration_file:$.: end-language mismatch: $lang\n"
+           if $lang ne lc $words[0];
+       }
+      elsif ($type eq 'args:')
+       {
+         push @{$language{$lang}}, @words;
+       }
+      else
+       {
+         die "$me: $configuration_file:$.: unknown directive: $type\n";
+       }
+    }
+}
+
+
 # parse_args ()
 # -------------
 # Process any command line arguments.
@@ -444,18 +506,36 @@ sub parse_args ()
 {
   my @trace;

-  Getopt::Long::config ("bundling");
+  # We want to look for the early options, which should not be found
+  # in the configuration file.  Prepend to the user arguments.
+  Getopt::Long::Configure ("bundling", "pass_through");
+  GetOptions (
+             "h|help"     => \&print_usage,
+             "V|version"  => \&print_version,
+
+             "l|language=s"  => \$language,
+             "v|verbose"     => \$verbose,
+             "d|debug"       => \$debug,
+            )
+    or exit 1;
+  Getopt::Long::Configure ("defaults");
+  unshift @ARGV, @{$language{$language}}
+    if $language;
+
+  verbose "arguments: @ARGV\n"
+    if $debug;
+
+  # Process the arguments for real this time.
+  Getopt::Long::Configure ("bundling");
   GetOptions
     (
      # Operation modes:
-     "h|help"       => \&print_usage,
-     "V|version"    => \&print_version,
-     "v|verbose"    => \$verbose,
-     "d|debug"      => \$debug,
      "o|output=s"   => \$output,
      "normalize"    => \$normalize,
      "f|force"      => \$force,
      "W|warnings=s" => address@hidden,
+     "m|mode=s"     => \$mode,
+     "M|melt"       => \$melt,

      # Library directories:
      "I|include=s" => address@hidden,
@@ -495,6 +575,9 @@ sub parse_args ()
   die "$me: the first file only can be frozen\n"
     if grep { /\.m4f/ } @ARGV[1 .. $#ARGV];

+  $ARGV[0] =~ s/\.m4f$/.m4/
+    if $melt;
+
   # We don't want to depend upon m4's --include to find the top level
   # files.  Try to get a canonical name, as it's part of the key for caching.
   for (my $i = 0; $i < $#ARGV; ++$i)
@@ -573,8 +656,19 @@ sub handle_output ($$)
   verbose "allowed   tokens: $allowed";

   # Read the (cached) raw M4 output, produce the actual result.
-  my $out = new IO::File (">$output")
-    or die "$me: cannot create $output: $!\n";
+  # We have to use the 2nd arg to have IO::File honor the third, but
+  # then stdout is to be handled by hand :(.
+  my $out = new IO::File;
+  if ($output eq '-')
+    {
+      $out->fdopen (fileno (STDOUT), "w");
+    }
+  else
+    {
+      $out->open($output, O_CREAT | O_WRONLY | O_TRUNC, oct ($mode))
+    }
+  die "$me: cannot create $output: $!\n"
+    unless $out;
   my $in = new IO::File ($ocache . $req->id)
     or die "$me: cannot read $ocache" . $req->id . ": $!\n";

@@ -956,6 +1050,7 @@ sub up_to_date_p ($)
 ## -------------- ##

 mktmpdir ('t4');
+load_configuration;
 parse_args;

 # We need our cache directory.
Index: lib/Makefile.am
--- lib/Makefile.am Wed, 01 Aug 2001 23:34:52 +0200 akim
+++ lib/Makefile.am Mon, 27 Aug 2001 14:20:02 +0200 akim
@@ -1,3 +1,28 @@
 ## Process this file with automake to create Makefile.in

 SUBDIRS = Autom4te m4sugar autoconf autotest autoscan
+nodist_pkgdata_DATA = autom4te.cfg
+EXTRA_DIST = autom4te.in
+
+edit = sed \
+       -e 's,@SHELL\@,$(SHELL),g' \
+       -e 's,@PERL\@,$(PERL),g' \
+       -e 's,@bindir\@,$(bindir),g' \
+       -e 's,@datadir\@,$(pkgdatadir),g' \
+       -e 's,@prefix\@,$(prefix),g' \
+       -e 's,@autoconf-name\@,'`echo autoconf | sed '$(transform)'`',g' \
+       -e 's,@autoheader-name\@,'`echo autoheader | sed '$(transform)'`',g' \
+       -e 's,@autom4te-name\@,'`echo autom4te | sed '$(transform)'`',g' \
+       -e 's,@M4\@,$(M4),g' \
+       -e 's,@AWK\@,$(AWK),g' \
+       -e 's,@VERSION\@,$(VERSION),g' \
+       -e 's,@PACKAGE_NAME\@,$(PACKAGE_NAME),g'
+
+# All the files below depend on configure.ac so that they are rebuilt
+# when the Autoconf version changes. Unfortunately, suffix rules
+# cannot have additional dependencies, so we have to use explicit rules.
+
+autom4te.cfg: $(top_srcdir)/configure.ac $(srcdir)/autom4te.in
+       rm -f autom4te.cfg autom4te.tmp
+       $(edit) $(srcdir)/autom4te.in >autom4te.tmp
+       mv autom4te.tmp autom4te.cfg
Index: tests/Makefile.am
--- tests/Makefile.am Fri, 24 Aug 2001 17:37:28 +0200 akim
+++ tests/Makefile.am Mon, 27 Aug 2001 17:08:18 +0200 akim
@@ -45,18 +45,16 @@
 check-local: atconfig atlocal testsuite
        $(SHELL) testsuite

-AUTOM4TE = autom4te_perllibdir='$(top_srcdir)/lib' ../bin/autom4te
+AUTOM4TE = AC_MACRODIR='$(top_srcdir)/lib' ../bin/autom4te
 testsuite: $(top_srcdir)/lib/m4sugar/m4sugar.m4 \
           $(top_srcdir)/lib/m4sugar/m4sh.m4 \
            $(top_srcdir)/lib/autotest/autotest.m4 \
            $(top_srcdir)/lib/autotest/general.m4 \
           atspecific.m4 \
            $(SUITE)
-       $(AUTOM4TE) -I $(srcdir) -I $(top_srcdir)/lib \
-         autotest/autotest.m4 suite.at -o address@hidden
-       chmod +x address@hidden
+       $(AUTOM4TE) --language=autotest -I $(srcdir) -I $(top_srcdir)/lib \
+         suite.at -o address@hidden
        mv address@hidden $@
-

 # The files which contains macro we check for syntax.  Don't use $(top_srcdir)
 # here since below we explicitly `cd' to $srcdir.  As for the dependencies,
Index: tests/atlocal.in
--- tests/atlocal.in Sun, 19 Aug 2001 13:08:48 +0200 akim
+++ tests/atlocal.in Mon, 27 Aug 2001 16:47:23 +0200 akim
@@ -12,5 +12,5 @@
 # autoconf which runs autom4te) because by themselves, they try to use
 # subtools from the same directory (i.e., foo/autoheader will run
 # foo/autoconf etc.).
-autom4te_perllibdir=$top_srcdir/lib
-export autom4te_perllibdir
+AC_MACRODIR=$top_srcdir/lib
+export AC_MACRODIR
Index: lib/autom4te.in
--- lib/autom4te.in Mon, 27 Aug 2001 17:17:40 +0200 akim
+++ lib/autom4te.in Mon, 27 Aug 2001 17:12:09 +0200 akim
@@ -0,0 +1,136 @@
+# Definition of macro sets.                              -*- Makefile -*-
+#
+# Copyright 2001 Free Software Foundation, Inc.
+#
+# This file is part of GNU Autoconf.
+#
+# GNU Autoconf is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# GNU Autoconf is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with autoconf; see the file COPYING.  If not, write to
+# the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+# Boston, MA 02111-1307, USA.
+
+## ---------- ##
+## Autoconf.  ##
+## ---------- ##
+
+begin-language: "Autoconf"
+# patterns: "*.ac"
+# patterns: "configure.in"
+args: --include @datadir@
+args: autoconf/autoconf.m4f
+args: --mode 777
+args: --warning syntax
+args: --normalize
+# Wanted by autoheader
+args: --preselect AC_CONFIG_HEADERS
+args: --preselect AH_OUTPUT
+args: --preselect AC_DEFINE_TRACE_LITERAL
+# Wanted by Automake
+args: --preselect AC_LIBSOURCE
+args: --preselect AC_SUBST
+# Wanted by autoscan
+args: --preselect AC_CHECK_FUNCS
+args: --preselect AC_CHECK_HEADERS
+args: --preselect AC_CHECK_LIB
+args: --preselect AC_CHECK_TYPES
+args: --preselect AC_C_CONST
+args: --preselect AC_C_INLINE
+args: --preselect AC_DECL_SYS_SIGLIST
+args: --preselect AC_FUNC_ALLOCA
+args: --preselect AC_FUNC_CHOWN
+args: --preselect AC_FUNC_ERROR_AT_LINE
+args: --preselect AC_FUNC_FNMATCH
+args: --preselect AC_FUNC_FORK
+args: --preselect AC_FUNC_FSEEKO
+args: --preselect AC_FUNC_GETGROUPS
+args: --preselect AC_FUNC_GETLOADAVG
+args: --preselect AC_FUNC_GETPGRP
+args: --preselect AC_FUNC_LSTAT
+args: --preselect AC_FUNC_MALLOC
+args: --preselect AC_FUNC_MEMCMP
+args: --preselect AC_FUNC_MKTIME
+args: --preselect AC_FUNC_MMAP
+args: --preselect AC_FUNC_OBSTACK
+args: --preselect AC_FUNC_SETPGRP
+args: --preselect AC_FUNC_SETVBUF_REVERSED
+args: --preselect AC_FUNC_STAT
+args: --preselect AC_FUNC_STRCOLL
+args: --preselect AC_FUNC_STRERROR_R
+args: --preselect AC_FUNC_STRFTIME
+args: --preselect AC_FUNC_STRTOD
+args: --preselect AC_FUNC_UTIME_NULL
+args: --preselect AC_FUNC_VPRINTF
+args: --preselect AC_FUNC_WAIT3
+args: --preselect AC_HEADER_DIRENT
+args: --preselect AC_HEADER_MAJOR
+args: --preselect AC_HEADER_STAT
+args: --preselect AC_HEADER_STDC
+args: --preselect AC_HEADER_SYS_WAIT
+args: --preselect AC_HEADER_TIME
+args: --preselect AC_PATH_X
+args: --preselect AC_PROG_AWK
+args: --preselect AC_PROG_CC
+args: --preselect AC_PROG_CPP
+args: --preselect AC_PROG_CXX
+args: --preselect AC_PROG_GCC_TRADITIONAL
+args: --preselect AC_PROG_INSTALL
+args: --preselect AC_PROG_LEX
+args: --preselect AC_PROG_LN_S
+args: --preselect AC_PROG_MAKE_SET
+args: --preselect AC_PROG_RANLIB
+args: --preselect AC_PROG_YACC
+args: --preselect AC_STRUCT_ST_BLOCKS
+args: --preselect AC_STRUCT_TIMEZONE
+args: --preselect AC_STRUCT_TM
+args: --preselect AC_TYPE_MODE_T
+args: --preselect AC_TYPE_OFF_T
+args: --preselect AC_TYPE_PID_T
+args: --preselect AC_TYPE_SIGNAL
+args: --preselect AC_TYPE_SIZE_T
+args: --preselect AC_TYPE_UID_T
+end-language: "Autoconf"
+
+## -------- ##
+## Autotest ##
+## -------- ##
+
+begin-language: "Autotest"
+# patterns: *.at
+args: --include @datadir@
+args: autotest/autotest.m4
+args: --mode 777
+args: --warning syntax
+end-language: "Autotest"
+
+## ---- ##
+## M4sh ##
+## ---- ##
+
+begin-language: "M4sh"
+# patterns: *.as
+args: --include @datadir@
+args: m4sugar/m4sh.m4
+args: --mode 777
+args: --warning syntax
+end-language: "M4sh"
+
+## ------- ##
+## M4sugar ##
+## ------- ##
+
+begin-language: "M4sugar"
+# patterns: *.msh
+args: --include @datadir@
+args: m4sugar/m4sugar.m4
+args: --warning syntax
+end-language: "M4sugar"