Re: [Automake-NG] typo whitelisting, and Automake-NG vs. GNU make runtime

[Stefano Lattarini]

On 08/22/2012 12:32 PM, Paolo Bonzini wrote:
> So I took a closer look at the whitelisting problem that was reported
> in GNU Smalltalk.
> 
> The piece of code that was removed in Automake-NG is:
> 
>   foreach my $primary ('SOURCES', 'LIBADD', 'LDADD', 'LDFLAGS', 
> 'DEPENDENCIES')
>     {
>       foreach my $var (variables $primary)
>         {
>           my $varname = $var->name;
>           # A configure variable is always legitimate.
>           next if exists $configure_vars{$varname};
> 
>           for my $cond ($var->conditions->conds)
>             {
>               $varname =~ 
> /^(?:EXTRA_)?(?:nobase_)?(?:dist_|nodist_)?(.*)_[[:alnum:]]+$/;
>               msg_var ('syntax', $var, "variable '$varname' is defined but no"
>                        . " program or\nlibrary has '$1' as canonical name"
>                        . " (possible typo)")
>                 unless $var->rdef ($cond)->seen;
>             }
>         }
> 
> 
> Note the
> 
>           # A configure variable is always legitimate.
>           next if exists $configure_vars{$varname};
> 
> part in particular.  Going back to the commit that introduced it, in
> 54ad2a6, the comment was even more expressive:
> 
>           # Note that a configure variable is always legitimate.
>           # It is natural to name such variables after the
>           # primary, so we explicitly allow it.
> 
> Now this suggests that indeed the new behavior of Automake-NG _would_ be
> "naughty and annoying" (quoting from Stefano).  Indeed there are cases
> where the new behavior would catch bugs, but are they enough overall?
> 
> 
> Generalizing the above, the point is that there _is_ a loss of information
> going from Automake runtime to GNU Make runtime.
>
Yes, unavoidable sometimes.  Though not in this case: we might simply add
all the relevant AC_SUBST to AM_VARTYPOS_WHITELIST.

> Therefore, I don't see
> the point in moving the check from Automake runtime to GNU make runtime.
>
How would you diagnose a typo in here at Automake runtime?

   bin_PROGRAMS = $(call user-func,args)
   bin_PROGRAMS += $(if $(ON-CYGWIN),baz)

   ifdef ON-CYGWIN
     # Oops, this was meant to be baz_LDFLAGS!
     bar_LDFLAGS = -lextra
   endif

It's true that this currently (and sadly) wouldn't work anyway, but that
is due to the limitations inherited by mainline Automake, not to some
fundamental impossibility; so I'd like to be able to make it work in the
future (the same way we made TEST_EXTENSIONS completely processed at make
runtime).  So we need to be prepared to the fact that a lot of information
will no longer be available at automake runtime.

For more information, refer to the commit that actually moved the
typo warnings from automake runtime to make runtime:

    commit 67d61020ff23f630bfa47bf6eec5036ecad5de74
    Author:     Stefano Lattarini <address@hidden>
    AuthorDate: Tue Jun 5 13:49:04 2012 +0200

    [ng] warns: typos in _SOURCES etc. reported at make runtime

    For example, on an input:

        bin_PROGRAMS = frob
        forb_SOURCES = main.c

    a proper warning should be given, since either 'forb' or 'frob' is likely a
    typo.  Mainline Automake gives the warning at automake runtime, while with
    this change, the warnings will be moved at make runtime.  This will allow
    us to warn about variables like '_DEPENDENCIES', whose definitions can
    be unavailable (or unanalyzable) at automake runtime (check about those
    variables had to be disabled in commit 'v1.12-295-g9a5f837' of 2012-05-22,
    "[ng] warns: don't report possible issues with '_DEPENDENCIES' variables",
    to avoid spurious errors).

> The tasks of Automake are multiple:
> 
> 1) gather information from configure.ac and merge it with Makefile.am
> files (options, subst-ed variables, etc.)
>
And this ability should be preserved in Automake-NG.

> 2) resolve conditionals
>
Ideally, once Automake-NG is powerful enough, Automake-time conditional
would disappear -- you should be able to use GNU make builtins like
'$(if ...)', 'ifeq' and 'ifdef' instead.  That will be quite difficult
to do (something for Automake-NG 2.0 probably), but is worth keeping in
mind that we are working towards that goal, and acting accordingly from
right now.

> 3) gather primaries, and select the appropriate snippets into the "rules"
> section of Makefile.in (roughly speaking)
>
Ideally, once Automake-NG is "completed", this too should be done completely
at make runtime (refer to the sadly-quiescent-if-not-downright-dead Quagmire
project for examples: <http://tromey.com/blog/?p=401>).

> 4) apply transformations to the snippets
>
This too will disappear eventually, to be replaced with proper use of $(eval)
and user-defined functions.  To get the gist of it, take a look at the
implementation of the parallel-tests harness in Automake-NG, in the file
'lib/am/parallel-tests.mk': that file is included *verbatim* in the generated
Makefile.in (well, almost: '##' comments still have to be stripped of course).

> 5) do semantic checks
>
Some of them (e.g., those involving only configure.ac) will be kept at
automake runtime; a lot will be moved at make runtime; some (I hope very
few) will just have to be dropped.  An unavoidable price to pay IMO.

> For each of this, the amount of code that can be moved to GNU make runtime
> is different.  For (1) it is obviously "zero",
>
Yep.

> and for (3) I suppose it is close to zero as well.
>
Disagree.  I actually want it to be 100% eventually :-)

> (2) could probably be simplified somehow, with
>     hardly any effect on the cost of actually building with GNU make.
>
True, but remember that the final objective is to completely get rid of
it, favouring the GNU make "equivalents" (ifeq, ifdef, $(if ...)).

> (4) can be simplified a lot, with some extra cost imposed on GNU make,
> but it should really be bearable.  In fact, simplifying this complex, and
> little known, part of Automake is the main benefit I see in requiring
> Automake-NG.
>
I agree this is the best starting point, and the simplest part to tackle
(and most of my changes have in fact worked in that direction).  But I
assure you this part of the Automake code base is actually one of the
simplest ans clearest ones!  The changes done in this area are mostly
useful as a step toward the overhaul of (2) and (3), as well as a way
to give the user the power to use more GNU make features in its input
Makefile.am (e.g., using wildcards or user-defined functions in the
definition of stuff like $(TESTS) and $(info_TEXINFOS)).

> For (5), it _may_ be possible (using things like .VARIABLES) to move the
> checks to GNU make.  But I don't really see the point.
>
This I should have explained above already.

> You want errors to be diagnosed as soon as possible, and as precisely
> as possible.
>
Yes -- as long as that is possible and doesn't impede further developments.

> Regarding quality, Perl can do the task better than Make,
>
But I envision a day when Automake-NG will be no more than a few hundred
lines of perl, mostly dedicated at tracing stuff in 'configure.ac', and
setting up few variable definitions accordingly.  No more incomplete, messy,
half-unspecified parsing and tweaking of a subset of make syntax there!
Let GNU make parse GNU make input -- it's the only sane way ahead.

> just like it can
> do it better than M4.  And Automake can also do it better than the generated
> Makefiles, because it has more information coming from tasks (1)
>
And this can be done, and will continue to be.

> and (2) in the list above.
>
Which I want to see disappeared some day, though (admittedly, that day is
still far away down the road, but I'm looking forward at it nonetheless).

> Regarding early diagnosis, I want "sh bootstrap.sh" to find all problems in
> my build system, *before* I embark into the expensive "./configure && make".
>
It's nice when it's possible of course, but if you want more "dynamicity" and
flexibility in your language (as I want in Automake-2.0), you'll have to forgo
some early warnings, sadly.  Just think Python vs. C -- we are moving from C
towards python; we'll earn a great deal in power and flexibility, and loose
a good deal in static checking.  Not a good thing, of course, but a necessary
trade-off.

> One of the reasons why the C compiler tries to warn about implicitly-defined
> functions, even though C89 does not require that, is that it diagnoses typos
> well before the final link.
> 
> Thoughts?
>
Yes, see above ;-)

Thanks,
  Stefano