Re: extending automake

[Brian Dessent]

Bob Rossi wrote:

> EXTRA_DIST = foo.xml
> 
> BUILT_SOURCES = foo.h foo.cc
> 
> foo.h foo.cc: $(srcdir)/foo.xml $(top_srcdir)/gen.py
>   python $(abs_top_srcdir)/gen.py $(srcdir)/foo.xml

This is not good.  When you write:

target1 target2: prereq
        recipe

It is really shorthand for:

target1: prereq
        recipe

target2: prereq
        recipe

You are not actually expressing to make that running 'recipe' generates
both target1 and target2, so there is a chance that make will needlessly
run 'recipe' twice.  Additionally, this is not parallel-make safe since
make will spawn two simultaneous instances of 'recipe' which will
compete to write to the same two output files.  There is an entire
section in the Automake manual (27.9, "Handling Tools that Produce Many
Outputs") dealing with this situation that's worth reading.

> I would like to automake this by writing a wrapper like,
>   AM_GEN(foo.xml)

I'm not sure that configure-time is where you want to deal with this.

> Is this possible? If so, is there some documentation on how to do it?
> I looked through the manual and didn't see it, or just missed it.

Remember that you can write arbitrary Makefile syntax in Makefile.am and
it is simply copied through to the output file, so in that sense you
don't have to use any special syntax to extend automake, just pretend
it's not there.

I see several potential methods of tackling this (both untested):

Method A, which would require the use of GNU make:

PYXMLFILES = foo.xml bar.xml baz.xml
EXTRA_DIST += $(PYXMLFILES)
BUILT_SOURCES += $(PYXMLFILES:.xml=.h) $(PYXMLFILES:.xml=.cc)

%.cc %.h: %.xml $(top_srcdir)/gen.py
        python $(word 2,$^) $<

Notes:
 - In general you should not use $(srcdir)/filename when referring to
targets or prerequisites in a Makefile.  That is the whole point of
VPATH -- make will automatically look in $srcdir to find things, whether
that means the current directory or somewhere else entirely.  The only
thing to remember is that when referring to the corresponding file in
the recipe, you should use automatic variables instead of specifying the
filename a second time.  That way, make substitutes the path from where
it was found from VPATH.  Or stated differently: don't duplicate
filenames from the target or prerequisite list in the recipe, refer to
them through $@, $<, $^, etc.

 - Writing a pattern rule with multiple targets (%.a %.b: %.c) is
different than writing a standard rule with multiple targets, in that
this form *does* tell make that both %.a and %.b are generated by
running 'recipe', so it should be parallel-safe.  But again this is GNU
make specific.

Method B, not GNU make specific, using a single witness stamp along the
lines of what is suggested in the manual:

PYXMLFILES = foo.xml bar.xml baz.xml
EXTRA_DIST += $(PYXMLFILES)
BUILT_SOURCES += $(PYXMLFILES:.xml=.h) $(PYXMLFILES:.xml=.cc)

pyxml-stamp: $(PYXMLFILES)
        @rm -f pyxml-tmp
        @touch pyxml-tmp
        (set -e; for F in $<; do \
          python $(abs_top_srcdir)/gen.py $$F; \
        done)
        @mv -f pyxml-tmp $@

$(PYXMLFILES:.xml=.h) $(PYXMLFILES:.xml=.cc): pyxml-stamp
        @if test -f $@; then :; else \
          rm -f $<; \
          $(MAKE) $(AM_MAKEFLAGS) $<; \
        fi

Notes:
 - This should be parallel-make safe due to the single stamp file,
except for the contrived situation mentioned in the manual where one or
more generated file is removed without also removing the stamp.
 - But also due to a single stamp file representing all .xml files,
parallel make won't be able to run multiple python invocations in
parallel, and if one .xml file is modified they will all be regenerated.

Brian